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1.  Introduction 


The  term  bandwidth  or  throughput,  as  used  in  computing  vernacular,  expresses  the 
overall  physical  constraints  of  a  device  to  process  and  deliver  data.  Despite  the  brain’s  ability  to 
exquisitely  cope  with  the  ubiquity  of  data  supplied  through  the  various  sensory  channels,  it  is 
generally  accepted  that,  as  with  any  information-processing  device,  the  brain  is  not  without  its 
processing  limits  (Broadbent,  1958).  Despite  this  observation,  technology  has  often  been 
designed  to  augment  biological  capacity.  For  example,  the  abacus,  slide  rule,  and  computer  each 
illustrate  technologies  that  have  supplanted  humans’  natural  computation  abilities.  Indeed,  there 
are  ample  examples  how  the  emergence  of  new  technologies  have  improved  human  cognition. 

The  miniaturization  of  digital  technology  has  also  permitted  the  global  interconnection  of 
knowledge  and  real-time  exchange  of  ideas.  However,  this  domain  of  technological  achievement 
is  currently  lacking  the  sophistication  for  sensitivity  to  environmentally  predefined  hierarchical 
task  requirements.  Consequently,  the  result  is  an  environment  where  technological  innovation 
effectively  becomes  the  prima  factor  reducing  an  operators’  already  limited  cognitive  capacity. 
For  example,  the  eco-system  of  personal  digital  technologies  (e.g.,  cell  phones)  exemplifies  the 
many  problems  posed  by  task  incompatibility  (attending  to  the  road  whilst  toggling  the  DVD 
player  contained  in  the  dashboard).  Similarly,  understanding  the  human  factors  associated  with 
the  novel  integration  of  systems  and  their  collective  impact  on  the  capacity  limits  of  the  human 
operator  are  likely  to  reduce  conditions  for  error,  or  stated  more  simply,  as  reductions  in  primary 
task  performance. 

The  manifestation  of  capacity  limits  can  be  operationalized  as  the  cognitive  workload  and 
stress  associated  with  task  performance.  To  date,  there  has  been  limited  theoretical  or  empirical 
research  to  articulate  the  factors  that  influence  the  complete  link  between  performance  and 
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perceived  workload.  Given  that  different  task  domains  produce  different  perfonnance-workload 

relations,  a  central  question  emerges:  what  factors  drive  subjective  measures  of  workload  and 

what  information  do  these  measures  provide  that  are  not  intrinsic  to  performance  measures? 

Such  relations  provide  only  on  single  scalar  link  between  multiple  faces  of  performance.  Each  of 

these  issues  are  addressed  in  the  present  context  to  better  understand  how  the  integration  of 

digital  collaborative  tools  into  command  and  control  (C2)  environments  effect  operator  mental 

workload.  While  it  is  important  to  understand  the  benefits  of  collaborative  tools  in  C2  domains, 

the  primary  purpose  of  this  project  was  to  evaluate  the  consequences  of  incorporating 

collaborative  tools  as  a  secondary  tasking  tool.  In  the  following  report,  we  address  the  major 

technical  challenges  involved  in  this  pursuit,  including  a  summary  of  relevant  research, 

underlying  theoretical  framework,  and  potential  implementations  to  quantify  the 

multidimensionality  of  workload  with  specific  a  locus  of  interest  towards  CT  in  C2  operations. 

Network-centricity,  as  stipulated  by  the  Department  of  Defense  (DOD  REFS),  has 

profoundly  contributed  to  the  rapid  entry  of  digital  technologies  into  all  echelons  military  life 

from  the  dismounted  soldier  to  the  battle  commander.  In  his  opening  paragraph  in  a  report  to  the 

United  States  Congress,  Wilson  describes  the  notion  of  Network-Centric  Warfare: 

Network  Centric  Warfare  (NCW)  is  a  key  component  of  DOD  planning  for 
transformation  of  the  military.  NCW  relies  on  computer  processing  power  and 
networked  communications  technology  to  provide  a  shared  awareness  of  the 
battle  space  for  U.S.  forces.  Proponents  say  that  a  shared  awareness  increases 
synergy  for  command  and  control,  resulting  in  superior  decision-making,  and  the 
ability  to  coordinate  complex  military  operations  over  long  distances  for  an 
overwhelming  war-fighting  advantage. 

As  a  result  of  the  DOD’s  NCW  transformation,  collaborative  technologies  (CTs),  in 
particular,  have  quickly  been  adopted  into  critical  military  operations.  Broadly  speaking,  CT 
refers  to  digital  platfonns  to  exchange  infonnation  between  two  or  more  people.  Such  tools  as 
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instant  messaging  (also  called  chat),  whiteboard,  image-  and  application-  sharing,  audio-  and 
video-  conferencing,  as  well  as  file  exchange  are  some  examples  falling  under  the  rubric  of 
digital  collaborative  tools.  The  military’s  incorporation  of  CTs  are  meant  to  improve  operator 
and  team  efficiency  by  serving  as  a  conduit  for  users  to  better  share,  organize,  and  process 
information,  which  may  significantly  improve  operator  productivity,  decision-making,  problem¬ 
solving,  and  team  situational  awareness. 

The  DOD’s  commitment  to  NCW  and  reliance  of  CTs  has  been  reaffirmed  in  such 
programs  as  the  Defense  Collaboration  Tool  Suite  (DCTS).  Accordingly  the  DCTS,  “program 
provides  Combatant  Commands  (COCOMS),  Services,  and  Defense  Agencies,  interoperable, 
real-time,  and  asynchronous  collaboration  capability  including  voice  and  video  conferencing, 
document  and  application  sharing,  instant  messaging,  and  whiteboard  capability  in  support  of 
defense  planning.”  Recently,  it  was  reported  that  the  Royal  Australian  Navy’s  Fleet  Information 
Systems  Support  Organization  (FISSO)  awarded  a  contract  for  development  of  a  collaborative 
suite  supporting  United  States  Special  Operations  Command  (USSOCOM)  who  have  made 
public  requests  for  technologies  that  include  network-based  collaborative  tools  (Brewin,  2006). 

1 . 1  Instant  Messaging  in  the  Military 

Instant  messaging  refers  to  communication  through  the  real-time  exchange  of  digital 
messages.  Digital  messaging  capabilities  have  long  been  a  sought  after  constituent  of  computing. 
One  of  the  earliest  programs  created  for  digital  message  exchange  called  “talk”  allowed  users  to 
connect  to  a  single  user  via  a  command  line  utility. 

In  the  early  evolution  of  mainstream  chat  applications  the  issue  of  chat  message 
interruptions  on  primary  task  performance  was  considered  by  Cutrell,  Czerwinski,  and  Horvitz 
(2000).  Participants  engaged  in  a  primary  web  search  task  and  a  subsequent  design  evaluation  of 
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the  graphic  design  quality  of  a  given  website.  For  analysis,  Cutrell  and  colleagues  categorized 
the  primary  task  into  three  discrete  phases.  A  planning  phase  where  participants  were  provided 
with  the  search  target  and  were  asked  to  construct  three  search  terms  prior  to  continuing  with  the 
search  and  evaluation  task.  An  execution  phase  where  the  search  terms  were  entered,  and  the 
evaluation  phase  where  participants  reviewed  the  query  results.  During  each  phase  participants 
were  interrupted  with  a  messaging  alert  where  they  were  prompted  to  open  the  message  window, 
reply,  and  click  an  ‘OK’  button  when  complete.  Messages  were  counterbalanced  such  that  in 
some  cases  they  were  relevant  to  the  primary  task,  or  alternatively,  irrelevant.  Cutrell  et.  al. 
observed  that  participants  were  significantly  slower  when  switching  to  the  instant  messaging 
window  during  the  execution  phase  of  the  task  (i.e.,  time  from  message  alert  to  opening  the 
message  window)  compared  to  interruptions  occurring  in  the  planning  and  evaluation  phases. 
Further,  a  small  but  significant  effect  was  observed  for  task  completion  times,  where  the 
evaluation  phase  took  longer  to  complete  compared  to  the  other  phases  (~  200  ms).  Message 
completion  time  as  well  as  the  time  to  resume  to  the  primary  task  was  significantly  longer  overall 
for  relevant  versus  irrelevant  messages.  The  study  suggests  that  while  message  interruptions 
generally  increase  task  demand,  that  message  interruptions  augment  certain  phases  of  a  task 
more  so  than  other  phases.  Whether  Cutrell  et  al’s  findings  have  direct  applicability  to  instant 
message  interruptions  in  command  and  control,  the  results  do  potentially  suggest  design 
recommendation  in  the  evolution  of  intelligent  systems  which  incorporate  chat. 

The  Navy  has  been  one  of  the  first  branches  of  the  military  to  integrate  digital 
collaborative  tools  into  daily  operations.  Thus,  the  majority  of  anecdotal  reports  or  published 
studies  have  evaluated  the  use  of  collaborative  tools  germane  to  Naval  operations.  In  one  study, 
Heacox,  Moore,  Morrison,  and  Yturralde  (2004)  had  approximately  two  hundred  respondents 
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surveyed  about  their  experience  using  chat.  Heacox  et  al  reported  that  respondents  say  they  use 
chat  6  to  7  days  per  week  for  at  least  7  hours  per  day,  that  nearly  a  quarter  of  respondents 
monitor  5  or  more  chat  sessions,  and  that  they  participate  in  multiple  chat  sessions.  Heacox  et 
al’s  report  was  principally  directed  at  development  of  chat  features  that  could  improve  user- 
interaction  with  new  chat  technologies;  however,  a  major  question  is  how  these  findings  impact 
task  performance.  While  there  is  an  advantage  of  using  chat  for  rapid  communication  of 
information,  it  is  unlikely  that  interface  designers  of  commercial  off  the  shelf  (COTS)  products 
pondered  use  cases  for  chat  usability  issues  that  could  emerge  from  engaging  in  chat  in  this 
manner.  While  it  is  important  to  consider  usability  issues  of  chat  clients  on  their  own,  an 
additional  tier  of  evaluation  must  consider  how  best  to  integrate  without  compromising  one’s 
principal  duties. 

Though  CTs  are  meant  to  replace  outdated  systems  for  information  exchange,  improve 
operator  efficiency,  or  to  simplify  complex  collaborative  tasks,  important  questions  remain  to  be 
answered  regarding  the  performance  and  workload  effects  of  integrating  network-based 
information  sharing  tools  into  contemporary  C2  operations.  During  Operation  Iraqi  Freedom,  for 
example,  instant  messaging  was  used  extensively  as  a  means  of  communication  for  the  Navy 
fleet  (Caterinicchia,  2003).  However,  anecdotal  evidence  suggested  that  a  number  of  unexpected 
issues  developed  as  a  result  of  this  form  of  communication.  Caterinicchia  (2003)  reported  that  a 
military  spokesperson  addressing  the  American  Society  of  Navel  Engineers’  Human  Systems 
Symposium  commented  that  the  volume  of  people  simultaneously  engaging  in  chat  as  a 
‘nightmare’. 

To  systematically  address  problems  associated  chat  communications,  Cummings  (2004) 
sought  to  address  both  instant  messaging  interface  issues  and  the  consequences  of  chat  on 
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primary  task  performance.  In  her  evaluation,  Cummings  used  a  Tactical  Tomahawk  Monitoring 
and  Retargeting  Task  that  required  participants  to  monitor  and  alter  a  missile’s  strike  position. 
During  the  primary  task,  participants  received  instant  messages  containing  either  requests  for 
information  or  status  about  the  current  task  conditions.  Messages  were  prioritized  based  on  their 
content.  Action  messages  required  participant’s  to  quickly  supply  information  to  a  superior. 
Health/Status  messages  were  supplied  by  the  missile  and  communicated  missile  position  and 
operational  status.  Information  messages  communicated  non-critical  information.  The  effect  of 
secondary  task  demand  (i.e.,  chat)  was  operationalized  using  time  to  reply  to  message  prompts 
and  response  accuracy. 

The  application  of  CTs  into  critical  C2  operations  could  arguably  be  labeled  a  success 

recently  demonstrated  during  the  Hurricane  Katrina  disaster.  As  reported  by  Moore  (2005),  the 

chat  feature  of  the  DCTS,  supported  through  the  Area  Security  Operations  Command  and 

Control  (ASOCC),  was  heavily  used  by  Naval  Emergency  Preparedness  Liaison  Officers 

(NEPLOs).  NEPLOs  primary  responsibility  during  the  Katrina  response  was  to  coordinate  with 

FEMA  and  other  emergency  agencies.  Moore  reported  that  the  chat  room  hosted  as  many  as  40 

simultaneous  connections,  with  approximately  4,500  instant  messages  exchanged  during  the 

response.  Early  chat  capabilities  were  reportedly  well-received: 

The  chat  tools  can  help  sailors  at  sea  make  repairs  more  quickly  when  they  need 
to  contact  technicians  on  shore  for  assistance.  Using  e-mail,  phone  and  other 
naval  message  systems  have  been  acceptable  methods,  but  they  can  cause 
downtime  while  technicians  wait  for  an  answer.  With  peer-to-peer  technology, 
personnel  at  sea  and  onshore  have  real-time  audio,  video  and  text  communication 
tools  to  help  them  find  solutions  Caterinicchia  (2001). 

In  addition  to  the  DOD  sponsored  collaborative  tools  such  as  the  DSTC,  a  recent 
adaptation  of  collaborative  tools  has  begun  to  permit  interoperability  among  the  Anny,  Navy, 
and  Air  Force  (Arnone,  2006).  The  IDM  or  Inter-Domain  Messaging  allows  for  secure 
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exchanges  between  these  military  branches  and  promotes  collaboration  and  information- sharing 
for  tactical  planning  and  knowledge  exchange.  Moreover,  the  US  Special  Operations  Command 
(USSOCOM)  announced  their  intent  in  securing  collaborative  tools  for  operational  planning 
during  the  Armed  Forces  Communications  and  Electronic  Association  (AFCEA;  Brewin,  2006). 
Primary  and  secondary  task  requirements  differentiate  chat  used  in  the  Katrina  disaster  versus 
chat  used  during  Operation  Iraqi  Freedom.  Those  users  employing  chat  to  communicate  during 
the  Katrina  recovery  efforts  arguably  were  using  chat  to  simply  communicate  without  the  burden 
of  another  task.  However,  one  would  expect  different  outcomes  as  a  function  of  using  chat  in 
these  different  domains.  While  both  environments  constitute  command  and  control,  and  both 
contain  elements  of  stress  due  to  the  magnitude  of  the  ongoing  situation,  it  is  not  surprising  that 
the  concatenation  of  chat  for  Navel  Operators  significantly  compromised  their  primary 
responsibilities. 

Applied  research  has  provided  significant  insight  into  some  of  the  human  integration 
challenges  for  realizing  the  efficient  exchange  of  information  via  chat  when  examined  in  concert 
with  other  primary  task  responsibilities.  However,  to  formally  approach  the  various  human 
factors  issues  associated  with  chat,  one  must  also  consider  the  global  infonnation  processing 
limitations  of  the  human  operator.  Therefore,  the  following  section  will  review  those  issues 
closely  associated  with  human  information  processing  constraints,  such  as  attentional  allocation, 
workload,  and  resource  capacity. 

1.2  Theoretical  Considerations 

Consistent  with  theories  of  selective  attention,  resource  theories  postulate  a  limited 
capacity  information  processing  system.  Thus,  the  ability  to  perform  a  given  task  is  conceptually 
related  to  the  amount  of  resources  available  to  meet  the  demands  of  that  task  (Kahneman,  1973; 
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Norman  &  Bobrow,  1975).  This  conjecture  does  not  preclude  multiple  critical  mental  operations 
from  occurring  concurrently  so  long  as  enough  resources  are  available  to  complete  each 
cognitive  action.  Resources,  in  this  view,  can  be  conceptualized  as  a  mental  fuel  that  is 
exhaustible,  thereby  leading  to  economic  metaphors  such  as  “paying”  attention  and  “investing” 
effort  in  tasks. 

Advocates  of  resource  theories  favor  one  of  two  lines  of  thought.  Early  resource 
proponents  advanced  a  single  limited  capacity  resource  pool  that  provides  the  capacity  for 
mental  operations  (Kahneman,  1973;  Norman  &  Bobrow,  1975).  Kahneman  (1973;  see  Figure 
1.1)  for  example,  argued  that  performance  on  a  task  or  a  group  of  tasks  depended  on  the  amount 
of  resources  allocated  to  each  activity. 

1.3  Mental  Workload  and  Performance 

The  human  brain  has  evolved  in  conditions  where  processing  constraints  have 
necessitated  neural  systems  that  seek  to  reduce  the  burden  of  a  data-rich  environment  by  filtering 
information  through  selective  attention.  The  brain  must  process  and  organize  the  voluminous 
data  collected  through  the  sensory  systems  into  a  meaningful  information  array  that  affords 
coordination  of  timely  response  to  novel  stimulus  contingencies  appearing  unpredictably  in 
space  and  asynchronously  in  time.  However,  the  systems  that  process  this  information  are 
inherently  limited  in  their  capacities  (Broadbent,  1958).  These  capacity  limits  are  manifested  in 
diverse  environments  as  the  cognitive  workload  and  stress  associated  with  task  performance.  To 
date,  there  has  been  limited  theoretical  or  empirical  research  to  articulate  the  factors  that 
influence  the  complete  link  between  perfonnance  and  perceived  workload.  Given  that  different 
task  domains  produce  different  performance-workload  relations,  a  central  question  emerges: 
what  factors  drive  subjective  measures  of  workload  and  what  information  do  these  measures 
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provide  that  are  not  intrinsic  to  performance  measures?  Such  relations  provide  only  on  single 
scalar  link  between  multiple  faces  of  performance. 

1 .4  Resource  Theories 

Resource  theories  emerged  after  the  fall  of  unitary  arousal  theory  (e.g.,  see  Hockey, 
1984;  Hockey,  Gaillard,  &  Coles,  1986).  Consistent  with  theories  of  selective  attention,  resource 
theories  postulate  a  limited  capacity  information  processing  system.  Thus,  the  ability  to  perform 
a  given  task  is  conceptually  related  to  the  amount  of  resources  available  to  meet  the  demands  of 
that  task  (Kahneman,  1973;  Norman  &  Bobrow,  1975).  This  conjecture  does  not  preclude 
multiple  critical  mental  operations  from  occurring  concurrently  so  long  as  enough  resources  are 
available  to  complete  each  cognitive  action.  Resources,  in  this  view,  can  be  conceptualized  as  a 
mental  ‘fuel’  that  is  exhaustible,  thereby  leading  to  economic  metaphors  such  as  “paying” 
attention  and  “investing”  effort  in  tasks. 

1.5  Single  resource  views 

Advocates  of  resource  theories  favor  one  of  two  lines  of  thought.  Early  resource 
proponents  advanced  a  single  limited  capacity  resource  pool  that  provides  the  capacity  for 
mental  operations  (Kahneman,  1973;  Norman  &  Bobrow,  1975).  Kahneman  (1973;  see  Figure  1) 
for  example,  argued  that  performance  on  a  task  or  a  group  of  tasks  depended  on  the  amount  of 
resources  allocated  to  each  activity. 
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Figure  1.1.  Kahneman ’s  single  resource  model  of  attention. 


Although  the  gross  capacity  of  the  single  resource  pool  is  fixed  its  net  capacity  in  part  a 
product  of  arousal.  At  center,  a  resource  allocation  mechanism  determines  how  much  of 
investment  to  allot  to  each  of  a  number  of  processing  activities  (at  bottom).  Such  factors  as 
momentary  intentions,  enduring  dispositions,  and  the  determination  of  capacity  demands 
influenced  the  allocation  of  resources  for  a  given  task.  If  resource  allocation  increases  while  task 
demand  remains  constant,  then  an  increase  in  performance  should  result. 

1.6  Multiple-resource  views 

In  contrast  to  the  view  that  the  brain  contains  only  a  single  pool  of  resources,  multiple 
resource  theories  argue  for  many  independent  limited  capacity  pools  that  could  operate 
independently  (Navon  &  Gopher,  1979;  Wickens,  1984;  2002).  Multiple  resource  theories  grew 
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to  accommodate  data  suggesting  that  performance  of  a  task  in  one  modality  (e.g.,  a  visual  task)  is 
virtually  unaffected  by  the  addition  of  a  task  occupying  another  modality  (e.g.,  an  auditory  task). 

In  the  most  widely  recognized  version  of  multiple  resource  theory  (see  Figure  1.2; 
revised  from  1984  model),  Wickens  (2002)  describes  a  four-dimensional  resource  model 
comprised  of  processing  stages  (perception,  cognition,  responding),  perceptual  modalities 
(visual,  spatial),  visual  channels  (focal,  ambient),  and  processing  codes  (spatial,  verbal).  This 
revised  version  of  multiple  resource  theory  evolved  from  an  earlier  three-dimensional  account 
(see  Wickens,  1984  of  a  detailed  description  of  each  of  these  dimensions),  which  did  not  include 
a  dimension  for  different  visual  channels  (i.e.,  focal,  ambient). 

Both  anecdotal  (reading  a  book  while  walking  down  a  hallway)  and  experimental  (the 
elucidation  of  brain  structures  specific  to  each  type)  evidence  makes  a  strong  case  for  disparate 
resources  between  these  two  types  of  visual  processing  (Wickens,  2002).  Focal  vision 
corresponds  to  foveal  vision  for  detailed  visual  processing,  while  ambient  vision  refers  to 
peripheral  vision.  In  both  variations,  Wickens  (2002)  argues  that  each  element  equates  to  discrete 
physiological  mechanisms  establishing  its  resource  independence  from  other  elements.  That  is, 
the  ability  to  drive  and  listen  to  the  radio,  read  and  listen  to  music,  speak  and  turn  on  the 
windshield  wipers,  can  all  co-occur  to  some  extent,  because  performance  of  each  relies  upon 
different  resource  streams.  Critics  of  multiple  resource  theory,  however,  argue  that  the  theory 
cannot  be  falsified  and  that  the  flexibility  afforded  by  the  concept  of  multiple  resources  is  easily 
adept  to  account  for  new  data  simply  by  adding  another  dimension  (Kantowitz,  1987). 
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Figure  1.2.  A  three-dimensional  illustration  ofWickens’  Multiple  Resource  Theory. 

1.7  Associations-Dissociations  of  Workload  and  Performance 

Theoretical  considerations  of  human  information  processing  capacity  limitations, 
considered  both  as  energetic  and  structural  limits  (e.g.,  see  Szalma  &  Hancock,  2002),  can  be 
utilized  for  better  understanding  workload,  performance,  and  the  link  between  them.  From  the 
perspective  of  energetic  models  of  performance,  workload  can  be  defined  as  the  difference 
between  operator  resources  and  the  cognitive  demands  imposed  by  a  task  or  situation  (e.g.,  see 
Wickens,  1992).  Therefore,  it  is  not  uncommon  for  researchers  to  elucidate  a  one-to-one 
mapping  between  workload  and  performance  for  a  given  task  by  simply  manipulating  task 
demand  (Cacioppo,  Tassinary,  &  Bernston,  2000).  Researchers  who  have  focused  primarily  on 
vigilance  tasks,  for  example,  generally  report  a  direct  and  consistent  link  between  performance 
and  mental  workload  such  that  increases  in  task  difficulty  simultaneously  induce  both 
performance  decrements  and  increased  workload  (Warm,  Dember,  Gluckman,  &  Hancock,  1991; 

Warm,  Dember,  &  Hancock,  1996;  but  see  also  Szalma  et  ah,  2004).  However,  if  a  task  is 
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manipulated  such  that  perceived  mental  workload  always  parallels  performance  while  decreases 
in  task  demand  always  produce  the  contrary  (i.e.,  complete  association),  the  diagnosticity  of 
subjective  measures  of  mental  workload  would  have  little  utility  except  in  circumstances  where 
it  is  impossible  to  measure  performance  itself  (see  Figure  1.3;  Hancock,  1996;  Parasuraman  & 
Hancock,  2001;  O’Donnell  &  Eggemeier,  1986;  Yeh  &  Wickens,  1988).  Conversely, 

investigators  of  multiple-task  performance  conditions  have  reported  dissociations  between 
perceived  workload  and  performance,  indicating  that  workload  is  not  simply  a  ubiquitous 
reflection  of  task  difficulty  (e.g.,  Yeh  &  Wickens,  1988). 

Limited  theoretical  and  empirical  research  has  been  conducted  to  articulate  the  factors 
that  contribute  to  performance- workload  associations  and  dissociations.  Given  that  different  task 
domains  produce  different  performance-workload  relations,  a  central  question  emerges:  What 
information  is  provided  by  subjective  measures  of  workload  that  is  not  provided  in  performance 
measures?  The  description  of  associations-dissociations  in  performance  as  descried  by 

Parasuraman  and  Hancock  (2001)  conceptually  formalizes  an  approach  to  evaluate  factors 
beyond  task  difficulty  that  may  contribute  to  explaining  the  variability  in  performance. 

1.8  Subjective  Measures  of  Mental  Workload 

Subjective  measures  of  mental  workload  are  an  important  tool  that  allows  easy 

assessment  of  the  perceived  task  demands  placed  upon  an  operator.  Numerous  psychometric 

tools  used  to  assess  mental  workload  have  been  developed  in  the  past  several  decades.  The 
currently  dominant  measures  of  perceived  workload  are  the  NASA-TLX  (Task  Load  Index;  Hart 
&  Staveland,  1988)  and  the  Subjective  Workload  Assessment  Technique  (SWAT;  Reid  & 
Nygren,  1988).  The  NASA-TLX  and  SWAT  are  multidimensional  scales  of  workload.  The 
NASA-TLX,  for  instance,  produces  a  global  workload  score  by  computing  a  weighted  average  of 
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ratings  on  the  following  six  subscales:  mental  demand,  physical  demand,  temporal  demand, 
performance,  effort,  and  frustration 


Workload 

Increase 

Association 

Performance 

Insensitivity 

Dissociation 

No  Change 

Workload 

Insensitivity 

Control 

Workload 

Insensitivity 

Decrease 

Dissociation 

Performance 

Insensitivity 

Association 

Degradation 

No  change 

Improvement 

Performance 

Figure  1.3  Matrix  of  performance  and  workload  association  and  dissociations  adapted  from 
Parasuraman,  R.  &  Hancock,  P.A.  (2001).  Adaptive  control  of  mental  workload.  In: 
P.A.  Hancock  and  P.A.  Desmond  (Eds.)  Stress,  workload  and  fatigue,  (pp.308) 
Erlbaum:  Mahwah,  NJ. 

1 .9  Psychophysiological  Measures  of  Mental  Workload 

While  subjective  measures  have  many  advantages  they  represent  but  one  output  system 
(subjective  and  verbal  report)  to  quantify  the  substrata  that  compose  the  multidimensionality  of 
workload.  Applying  a  three-systems  approach  to  understand  workload  (see  Lang,  1993), 
physiological  function  may  provide  critical  information  not  accessible  or  not  readily  assessed 
through  other  means.  Psychologist  and  neuroscientist  alike,  for  example,  have  mutually  benefited 
with  the  advent  of  neuroelectric  assessment  and  blood  oxygen  imaging  techniques.  Nevertheless, 
technological  innovation  is  no  substitute  for  weak  experimental  design.  In  the  context  of 
understanding  the  complexity  of  human-machine  systems,  it  is  sometimes  impractical  to 


incorporate  physiological  instrumentation  as  a  result  of  unreasonable  signal-to-noise 
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compromises,  logistical  difficulties,  or  obtrusiveness  with  primary  task  demands  (Kramer  & 
Weber,  2000).  Nevertheless,  as  with  any  task,  the  ability  to  assess  multiple  levels  of  analysis  has 
its  benefits,  specifically  when  assessing  constructs  that  may  be  difficult  to  measure  with 
conventional  methods. 

1.9.1  Event  Related  Potentials 

Psychophysiological  measures  have  been  successfully  used  to  investigate  a  number  of 
issues  as  they  pertain  the  human  mental  workload  (Parasuraman,  1990).  ERPs  refer  to  a  broad 
class  of  waveforms  that  occur  specifically  in  response  to  actual  or  anticipated  stimuli  (Andreassi, 
2000).  ERP’s  are  extracted  by  timelocking  the  raw  electrocortical  activity  to  some  temporal 
event.  ERPs  differ  from  frequency  derived  EEG  data  in  that,  the  latter  provides  spectral  specific 
information.  The  term  ERP  encapsulates  a  large  set  of  time-locked  cortically  derived  measures, 
for  example,  long-latency  potentials,  sensory  ERPs  (auditory,  visual,  somatosensory),  and 
steady-potential  shifts  (contingent  negative  variation,  readiness  potentials,  far-field  potentials), 
each  varying  in  their  latency  and  response  characteristics.  Because  of  signal-to-noise  issues, 
ERPs  are  quite  small,  and  therefore  multiple  trials  are  generally  averaged  to  remove  the 
background  EEG  activity  not  time-locked  to  the  stimulus  or  event.  Once  completed,  waveform 
summary  statistics  can  be  isolated  by  extracting  peak  amplitude  (positive  or  negative)  and  peak 
latency  (see  Fabiani  et  al.,  2000,  Picton  et  ah,  2000). 

ERPs  have  been  used  extensively  to  examine  human  information  processing,  with  latency 
and  amplitude  response  characteristics  thought  to  vary  with  the  attentional  requirements  of  a  task 
(for  a  comprehensive  review,  see  Birbaumer  et  al.,  1990;  Hillyard  &  Hansen,  1986).  For 
example,  both  the  N 1  and  P3  components  of  the  cortical  event-related  potential  (ERP)  have  been 
used  to  evaluate  mental  workload,  and  attention  in  general.  For  the  P3  (i.e.,  the  positive 
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deflection  that  occurs  at  approximately  300  ms),  a  typical  monotonic  relationship  is  often 
identified.  That  is,  as  workload  increases,  ERP  amplitude  evoked  in  response  to  a  secondary  task 
is  attenuated  (e.g.,  Kramer,  Wickens,  &  Donchin,  1985).  Because  ERPs  appear  to  index  the 
capacity  limitations  predicted  by  resource  models,  they  provide  a  unique  means  to  assess 
workload  at  the  earliest  stages  of  information  processing. 

1.9.2  Brain  Imaging  Techniques 

Unlike  EEG  that  has  excellent  temporal  but  poor  spatial  resolution,  both  positron 
emission  tomography  (PET),  and  functional  magnetic  resonance  imaging  have  good  spatial 
resolution  but  poor  temporal  resolution.  However,  both  of  these  techniques  have  been  used 
extensively  to  isolate  those  brain  areas  whose  activation  is  coincident  with  human  information 
processing.  Gruber  and  colleagues  (Gruber,  Muller,  Keil,  &  Elbert,  1999),  for  example,  found 
that  cerebral  blood  flow  (via  PET)  increases  in  specific  brain  areas  when  an  individual  attends  to 
a  specific  stimulus,  or  shifts  spatial  attention  to  visual  stimuli  in  varying  visual  fields.  PET 
evidence  has  indicated  that  the  superior  parietal  and  superior  frontal  cortex  activate  when 
attention  shifts  to  peripheral  locations  compared  to  when  attention  gaze  is  fixated  at  a  center 
location  (Corbetta,  Miezin,  Shulman,  &  Peterson  1993).  Similarly,  fMRI  evidence  indicates  that 
blood  oxygenation  in  specific  cortical  regions  increases  when  participants  attend  to  dynamic 
stimuli  as  compared  to  when  this  stimuli  is  ignored  (Baauchamp,  Cox,  &  DeYoe,  1997;  Haug, 
Baudewig,  &  Paulus,  1998).  It  may  be  assumed  that  attention  results  in  a  salient  magnification  of 
neuronal  activity  in  cortical  regions  or  pathways  that  are  associated  with  the  demands  that  are 
required  to  process  the  attended  stimulus  (Hillyard,  Vogel,  &  Luck,  1998). 
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1.9.3  Heart  Rate  Variability 

Like  brain  measurement  techniques,  measures  of  heart  rate  variability  (HRV)  have  been 
used  extensively  as  a  physiological  metric  for  mental  workload  (Nickel,  Friedhehn,  &  Ossietzky, 
2003).  Variability  of  the  heart  signal  refers  primarily  to  the  variance  associated  with  the  interbeat 
interval  as  indicated  by  each  R-wave  peak,  although  different  electrocardiographic  (ECG) 
components  are  also  employed  for  performing  medical  diagnoses.  Once  parsed  from  the  raw 
ECG,  interbeat  intervals  are  commonly  subjected  to  a  Fourier  transformation  to  obtain  the 
spectral  power  in  different  frequency  ranges.  For  workload,  what  is  termed  the  low  frequency 
band,  which  reflects  spectral  power  in  the  0.04  to  0.15  Hz  range  (LFB)  and  the  high  frequency 
band  0.15  to  0.4,  has  received  the  greatest  interest  because  these  range  appears  most  sensitive  to 
manipulations  of  stress  and  workload  (Appelhans  &  Luecken,  2006).  The  sensitivity  of  the  LFB 
and  HFB  ranges  to  task  induced  stress  is  partially  a  result  of  vagus  nerve  innervation  by 
parasympathetic  and  sympathetic  inputs.  The  right  vagus  projects  to  the  heart’s  sinoatrial  node, 
which  serves  as  the  heart’s  pacemaker.  Innervation  of  the  sinoatrial  node  produces  a  decrease  in 
heart  rate  (i.e.,  parasympathetic  input).  The  sinoatrial  node  also  receives  afferent  input  from  the 
sympathetic  system  via  spinal  nerves.  The  spectral  variability  is  a  product  of  the  slow  acting 
norepinephrine  from  sympathetic  inputs  versus  acetylcholine  inputs  via  the  parasympathetic 
system  (Appelhans  &  Luecken,  2006).  Spectral  power  in  the  low  frequency  band  would 
therefore  indicate  sympathetic  influences,  while  power  in  the  high  frequency  band  would 
indicate  parasympathetic  influences. 

1 .9.4  Pupillometry  and  Eye  Tracking 

Measurement  of  pupillary  changes  as  a  function  of  task  load  has  been  a  longstanding  tool 
for  exploring  numerous  psychological  and  cognitive  processes  (Andreassi,  2000;  Janisse,  1977). 
Such  measures  have  been  used  in  the  study  of  cognitive  effort  and  information  processing 
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extensively,  where  it  is  typical  to  observe  increases  in  pupil  dilation  as  a  function  of  processing 
load  (Beatty,  1982;  Granholm,  Asarnow,  Sarkin,  &  Dykes,  1996;  Hyona,  Tommola,  &  Alaja, 
1995).  Despite  its  many  uses,  however,  evaluation  of  pupillary  changes  to  complex  visual  stimuli 
presents  the  problem  of  effectively  parsing  psychological  modulation  from  core  pupillary 
function.  For  instance,  the  light  reflex  (i.e.,  regulating  the  amount  of  light  that  enters  the  eye)  and 
accommodation  response  (i.e.,  changing  the  curvature  of  the  lens  to  control  depth  of  field), 
among  others  (see  Tyron,  1975),  are  two  such  sources  of  variation  that  potentially  may  confound 
response  measurements.  Bradley  (2000)  has  addressed  this  issue  when  assessing  cardiac  function 
in  emotion,  where  the  primary  “homeostatic  and  metabolic  (p.  624)”  needs  of  the  body  can  mask 
affective  influence.  The  use  of  the  pupil  is  another  such  measure  that  can  easily  be  confounded  in 
this  regard,  yet,  its  sensitivity  to  task  demands  merits  its  role  in  the  psychophysiological  study  of 
workload  if  these  factors  are  considered.  In  addition  to  both  ERPs  and  pupillary  function,  eye¬ 
tracking  can  provide  important  information  regarding  the  allocation  of  attention  in  real  time. 

Eye  tracking  can  provide  important  insight  regarding  the  level  of  workload  by  revealing 
how  an  operator  interacts  with  his/her  environment  through  visual  information  extraction 
strategies  (Kramer  &  Weber,  2000).  Seagull  et  al.  (1999),  for  example,  developed  a  resource 
model  of  monitoring,  where  the  human  operator  builds  knowledge  representations  about  their 
environment.  The  pattern  of  monitoring,  however,  changes  depending  on  the  level  of  operator 
workload.  That  is,  as  workload  increases  monitoring  behaviors  decrease.  Beyond  monitoring 
behavior,  identifying  strategy  shifts  are  an  important  marker  of  workload  because  they  reflect  the 
amount  of  cognitive  effort  that  an  operator  gives  to  a  task.  By  evaluating  eye  movement  metrics 
during  collaborative  task  performance,  we  may  also  be  able  to  elucidate  operator  strategy  shifts 
that  may  not  be  observable  through  other  means  (Marshall,  Pearce,  Dickson,  2002). 
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1.10  Summary 

A  primary  goal  of  the  proposed  project  is  to  understand  how  introduction  of  collaborative 
technologies  affects  performance  and  workload  associated  with  C2  tasks.  Accomplishing  this 
goal  requires  multidimensional  measurement  of  workload  and  analysis  of  the  associations  and 
dissociations  that  occur  between  workload  and  task  performance  (Matthews,  2001;  O’Donnell  & 
Eggemeier,  1986).  To  this  end,  we  have  proposed  a  multi-modal  assessment  bridging  subjective, 
behavioral,  and  physiological  levels  of  analysis  to  provide  a  comprehensive  account  of  workload 
and  performance.  Each  methodology  (performance,  physiological  measurement,  and  subjective 
response)  offers  a  separate  vista  through  which  the  interaction  of  workload  and  human 
performance  can  be  viewed  and  therefore  provide  a  more  comprehensive  and  accurate 
assessment.  We  intend  to  transfer  the  techniques  and  developed  methodologies  derived  for  the 
measurement  of  the  response  of  the  individual  to  a  companion  method  aimed  at  providing  a 
comprehensive  analysis  of  the  operational  context.  Success  in  these  efforts  will  be  central  to  any 
success  of  an  overall  program  to  provide  training  and  simulation-based  support  to  modern 
soldiers. 

2.  Method 

2.1  Participants 

Participants  included  26  students  from  the  University  of  Central  Florida  who  were 
recruited  from  undergraduate  courses  in  exchange  for  a  small  amount  of  extra  course  credit.  Of 
those  participants,  1 1  were  Male  and  15  were  Female,  with  an  average  age  of  21.67  years  (SD  = 
6.17).  Most  participants  described  themselves  as  Caucasian  (n  =  18).  Other  racial  groups 
represented  in  the  sample  included  African-American  (n  =  3),  Hispanic  (n  =  4),  and  Asian  (n  = 

1).  The  majority  of  participants  were  Freshman  (n  =  15),  followed  by  Seniors  (n  =  6), 
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Sophomores  (n  =  3),  and  Juniors  (n  =  2).  Because  of  experimenter,  computer,  or  other  technical 
problems,  the  final  samples  sizes  were  the  following  for  each  dependent  variable:  DDD 
Performance  data  (n  =  22),  Heart  Rate  Variability  (n  =  23),  Self-Reported  Workload  (n  =  21), 
ERPs  (n  =  10),  Secondary  Task  Performance  (n  =  19). 

2.2  Materials 

Perceived  mental  workload  was  measured  by  the  NASA  Task  Load  Index  (NASA-TLX; 
Hart  &  Staveland,  1988).  The  NASA-TLX  is  an  instrument  that  gauges  six  dimensions  of 
perceived  workload  on  a  scale  of  0  to  100.  A  computerized  version  of  the  NASA-TLX  was 
created  for  the  experiment.  This  version  was  written  in  Lab  VIEW  8.0  and  presented  the  various 
NASA-TLX  dimensions  (i.e.,  Mental  Demand,  Physical  Demand,  Temporal  Demand, 
Performance,  Effort,  Lrustration)  to  participants  using  customized  sliders.  The  computerized 
version  also  presented  all  possible  pairwise  comparisons  randomized  for  each  participant. 

In  all  experiments  mood  states  was  measured  using  the  Dundee  State  Questionnaire 
(DSQ),  which  is  a  multidimensional  instrument  for  assessing  transient  states  associated  with 
mood,  arousal  and  fatigue  (Matthews  et  ah,  1999).  This  instrument  has  been  shown  to  be 
sensitive  in  investigating  the  effects  of  task  factors  on  operator  mood  state  that  can  potentially 
contribute  to  task  demands  (Grier,  et  al.,  2004;  Helton,  Dember,  Wann,  &  Matthews,  1999; 
Matthews,  et  al.,  1999;  Szalma  et  al.,  2004). 

Both  state  and  trait  anxiety  were  assessed  using  the  STAI  (State-Trait  Anxiety  Inventory, 
Spielberger  et  al.,  1983).  Each  dimension  of  the  STAI  (i.e.,  state  and  trait)  consists  of  a  20-item 
questionnaire,  which  ask  respondents  to  indicate  how  they  feel  about  a  particular  statement  using 
a  4-point  scale  anchored  at  the  extremes  with  1  (almost  never)  to  4  (almost  always).  High  scores 
reflect  greater  state/trait  anxiety.  STAI  scores  have  demonstrated  sound  psychometric  properties, 
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including  inter- item  consistency,  predictive  validity,  differential  stability  for  trait  anxiety  (i.e., 
test-retest  reliability  ~  .75),  and  divergent  validity  (Spielberger  et  al.,  1983). 

2.3  Apparatus 

The  experiment’s  synthetic  task  environment  (STE)  was  a  command  and  control  task 
developed  for  use  with  the  Dynamic  Distributed  Decision-Making  Testbed  (DDD;  Aptima,  Inc.). 
The  DDD  is  a  software  application  that  serves  as  a  platform  for  designing  uni-  and  multi-player 
task  scenarios.  These  task  scenarios  can  be  created  such  that  they  mimic  certain  characteristics 
common  to  real-world  command  and  control  domains. 

The  experiment  used  the  ‘Tanker  Scenario’  provided  by  our  AFRL  research  partner  that 
was  subsequently  modified  for  use  with  a  single  operator.  In  the  modified  version,  the  participant 
serves  the  dual  role  of  Battle  Commander  and  the  Strike  Operator.  The  scenario  requires  the 
operator  manage  a  number  of  air  (Refueling  Tanker,  Jet  Fighter)  and  ground  assets  (Base). 
Management  in  this  capacity  requires  that  the  operator  protect  assets  by  coordinating  attacks 
against  enemies,  refueling  jet  fighters,  and  launching  new  aircraft  when  necessary.  The  ‘Tanker 
Scenario’  allows  the  operator  to  view  the  entire  battle  space. 

The  DDD  testbed  is  presently  designed  for  use  with  Linux  (Red  Hat  9,  Fedora  Core  4). 
Because  the  secondary  task  required  integration  of  instrumentation  that  could  only  run  in  a 
Microsoft  Windows  environment,  the  following  testing  environment  was  assembled.  First, 
Parallels  Virtual  Machine  was  used  as  a  hypervisor  to  host  the  Linux  operating  system  (OS) 
within  Microsoft  Windows  XP.  Figure  2.1  shows  an  example  of  this  virtualization  on  using 
Apple  OSX  as  the  host  operating  system  platform  and  shows  RedHat  9  Linux  and  Microsoft 
Windows  XP  guest  OS’s  operating  concurrently  within  this  host.  This  configuration  offers  a 
great  degree  of  flexibility  given  that  multiple  RedHat  OS’s  can  be  cloned  and  configured  across 
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multiple  computers.  Figure  2.2  depicts  a  screenshot  of  the  actual  experimental  window.  The 
image  appears  smaller  because  the  screenshot  spans  two  monitors.  The  left  side  of  the  figure 
shows  the  DDD  task  window  running  in  Linux  The  right  side  of  the  figure  displays  the 
physiological  data  collection  software  running  in  Windows. 

The  Laboratory  testbed  used  three  monitors,  two  for  the  experimenter  and  the  other  for 
the  participant.  A  Belkin  monitor  replicator  was  used  to  duplicate  the  Linux  window  for  view 
both  by  the  participant  and  experimenter,  while  the  experimenter  used  the  other  monitor  display 
the  physiological  software.  The  Linux  window  was  recorded  for  the  each  condition  using 
Camtasia  Software  (Techsmith). 


Crab  File  Edit  Capture  Window  Help 
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Figure  2.1.  Example  of  the  virtualization  environment  showing  an  Apple  OSX  host  operating 
system,  running  two  guest  operating  systems  (RedHat  Linux  9  on  the  left  and 
Microsoft  Windows  XP  on  the  Right). 
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Figure  2.2.  Screenshot  shows  the  experimenter  operating  views.  The  DDD  testbed  (shown  on  the 
right)  was  recorded  by  screen  capturing  software  and  duplicated  to  the  participant  on 
another  monitor.  The  right  half  of  the  image  shows  the  physiological  data  collection 
software  and  the  digital  version  of  the  NASA-TLX. 


2.4  Physiological  Data  Collection 

All  physiological  measures  were  collected  using  Acknowledge  software  (3.7.8,  Biopac 

Systems,  Inc.,  Santa  Barbara,  CA)  that  was  used  to  interface  an  MP150  control  module  (12-bit 

A/D  converter;  Biopac  Systems,  Inc).  All  signals  were  sampled  at  1  kHz  for  the  entire  length  of 

each  task  condition.  Electrocardiographic  (ECG)  measurement  was  collected  using  Ag/AgCl 

electrode  leads  placed  on  the  left  and  right  middle  deltoid.  Raw  ECG  signals  were  analog  filtered 

with  a  passband  of  0.05  to  35  Hz  and  amplified  5000  times.  Electroencephalographic  (EEG) 

activity  was  assessed  at  three  Parietal  sites  (P3,  PZ,  P4)  via  an  electrode  (Sn)  cap  (Electro-cap 

International)  with  placement  specified  by  the  International  10-20  System  (Jasper,  1958).  Raw 

EEG  signals  were  filtered  with  a  passband  of  0.01  to  35  Hz  and  amplified  50,000  times.  All 

channels  were  referenced  to  a  physically  linked  ears  reference  configuration  using  two  tin 

electrodes  positioned  on  the  lobule  of  the  left  and  right  ear.  To  correct  for  ocular  artifacts, 

vertical  and  horizontal  eye  movement  was  monitored  via  an  electroocculogram  bioamplifier 
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(EOG)  using  Ag/AgCl  electrodes.  Horizontal  eye  movement  was  recorded  at  the  left  and  right 
outer  canthi,  and  vertical  eye  movements  were  recorded  from  positions  above  and  below  the  eye. 


Figure  2.3  illustrates  the  testbed  setup.  The  inclusion  of  the  eyetracker  is  illustrated  for  possible 
inclusion  for  additional  experimentation. 


Figure  2.3.  Diagram  illustrates  the  hardware  and  software  configuration  for  the  assembled 
testbed. 

2.5  Data  Reduction 

Dependent  measures  for  heart  rate  variability  were  obtained  using  the  following  method. 
Following  artifact  removal,  R-R  intervals  (Interbeat  Interval  -  IBI)  were  extracted  from  the 
filtered  and  amplified  ECG  signal.  To  extract  spectral  components,  these  values  were  then 
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subjected  to  Fast  Fourier  Transformation.  Once  complete,  peak  power  in  the  following  spectrums 
indices  were  derived  as  gross  dependent  measures  of  sympathetic  activation:  Very  Low 
Frequency  Band  (VLFB)  0  to  0.04  Hz;  Low  Frequency  Band  (LFB)  0.04  to  0.15  Hz,  High 
Frequency  Band  (HFB) 

To  isolate  dependent  measures  derived  from  EEG  recordings,  event-related  potentials 
(ERP)  were  derived  by  averaging  EEG  activity  100  ms  prior  and  500  ms  following  the  onset  of 
time-locked  to  the  onset  of  the  tone.  Once  averaged,  the  ERP  was  baseline  corrected  by 
subtracting  a  computed  average  of  the  100  ms  EEG  activity  from  the  pre-tone  baseline  from  each 
subsequent  point  in  the  waveform.  For  the  dependent  measures  of  interest,  the  peak  amplitude 
and  latency  of  the  N 1  and  P3  was  used  as  the  summary  statistics  for  later  analyses. 

2.6  Procedure 

Participants  visited  the  lab  on  two  separate  occasions.  On  the  first  visit,  participants  were 
required  to  read  and  sign  a  University  approved  informed  consent  form  (see  Appendix  A). 
Participants  were  then  asked  to  complete  the  STAI-Trait,  the  EPQ,  and  a  short  demographic 
questionnaire.  Once  complete,  participants  were  presented  with  the  DDD  training  materials  (see 
Appendix  B).  The  training  materials  were  presented  in  an  instructor-lead  fonnat,  where  the  basic 
operation  of  the  DDD  and  some  details  of  the  ‘Tanker  Scenario’  were  provided.  These  details 
included  such  information  as  the  participant’s  function  in  the  DDD  STE,  their  primary  mission 
objectives,  and  scenario  rules.  Participants  were  also  instructed  that  they  would  be  relaying 
latitude  and  longitude  coordinates  via  an  instant  messaging  interface  at  various  points  during  the 
experiment.  Once  final  questions  about  the  DDD  STE  were  addressed,  participants  then 
completed  the  NASA-TLX.  This  was  to  familiarize  participants  with  the  instrument  prior  to  the 
experimental  session.  The  participants  then  had  an  opportunity  to  use  the  DDD  in  a  simple  task 
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scenario.  The  scenario  was  a  modified  ‘Tanker  Scenario’,  where  a  single  base,  tanker,  and  jet 
tighter  served  as  the  primary  assets.  Participants  were  required  to  qualify  for  the  study  by 
completing  the  following  objectives:  (1)  Pursue  an  enemy,  (2)  Destroy  an  enemy,  (3)  Orbit  an 
aircraft,  (4)  Launch  an  aircraft  from  the  base,  (5)  Refuel  at  the  tanker.  Participants  were  then 
scheduled  for  the  experimental  session.  The  first  session  took  approximately  1  hour  and  30 
minutes  to  complete.  The  second  visit  consisted  of  the  experimental  session.  Participants  were 
asked  to  first  complete  the  STAI-State  and  the  DSSQ  pre-task  questionnaire.  The  electrodes  were 
then  affixed.  The  experiment  exposed  participants  to  two  task  Scenarios  consisting  of  (Easy, 
Difficult)  by  Message  (One  Latitude/Longitude  Coordinate,  Three  Latitude/Longitude 
Coordinates)  combinations  co  (i.e.,  four  total  conditions)  in  a  counterbalanced  arrangement 

3.  Results 

While  no  a  priori  hypothesis  were  made  regarding  performance  differences  between 
gender,  preliminary  evaluation  of  the  data  indicated  that  a  potential  confound  existed  when 
collapsing  results  across  gender.  While  workload  hypothesis  related  to  Gender  differences  as 
operators  engage  in  collaborative  dual-tasking  was  not  the  intended  emphasis  of  this 
investigation,  results  will  be  presented  such  that  Gender  will  be  introduced  as  a  factor  to  isolate 
the  variance  attributed  to  this  variable. 

3.1  DDD  Performance  Data 

Because  preliminary  examination  of  the  data  indicated  empirical  differences  with  respect 
to  primary  task  performance  and  gender,  primary  task  perfonnance  was  evaluated  using  a 
Multivariate  Analysis  of  Variance  (MANOVA)  across  the  primary/secondary  factor 
combinations  (Easy/One  Message,  Easy/Three  Message,  Difficult/One  Message,  Difficult/Three 
Message).  This  combination  yielded  a  Gender  (Male,  Female)  x  Primary  Task  Difficulty  (Easy, 
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Hard)  x  Message  Length  (One  Message,  Three  Messages)  with  repeated  measures  on  the  last  two 


factors.  The  following  dependent  variables  were  included  in  the  analysis:  Total  Enemies 


Destroyed,  Total  Attacks  in  Green,  Yellow,  and  Red  Zone,  Total  Fighters  Lost,  Total  Enemies  to 


Enter  the  Green,  Yellow,  or  Red  Zone,  the  total  time  in  the  Green,  Yellow,  and  Red  Zone,  Total 


Correct  Refuels,  Total  Incorrect  Refuels,  Time  From  Target  Identification  to  Target  Elimination, 
and  Aircraft  Lost  to  Fuel. 

The  omnibus  multivariate  interaction  for  Gender  x  Primary  Task  Difficulty  x  Message 
Length  only  approached  significance,  F  (15,  6)  =  3.49,  L  =  0.103,  p  =  .066.  Although  not 
statistically  significant,  this  interaction  was  further  examined  given  the  strong  linear  trend  and 
the  exploratory  nature  of  this  experiment.  Follow-up  univariate  Analysis  of  Variance  (ANOVA) 
tests  were  subsequently  considered  to  further  isolate  those  dependent  variables  contributing  to 
the  interaction.  Both  Total  Fighters  Lost  and  Total  Aircraft  Lost  to  Fuel  were  significant,  F  (1, 
20)  =  5.51,p  <  .05  andF(l,  20)  =  4.80,  p  <  .05,  respectively.  A  strong  linear  trend  was  observed 
for  the  Total  Enemies  to  Enter  the  Yellow  Zone,  F  (1,  20)  =  3.10,  p  =  .09.  Table  3.1  presents  the 
means  for  each  of  these  dependent  variables.  Post  hoc  analyses  were  only  considered  for  Total 
Fighters  Lost  and  Total  Aircraft  Lost  to  Fuel. 


Table  3. 1.  Gender  x  Primary  Task  Difficulty  x  Message  Cell  Means 


Fighters  Lost 

Aireaft  Lost  Fuel 

Enter  Yellow 

Gender 

Primary 

Task 

Messages 

Mean 

Std. 

Error 

Mean 

SE 

Mean 

SE 

Male 

Easy 

One 

0.000 

0.286 

0.222 

0.234 

0.222 

0.209 

Three 

0.111 

0.120 

0.556 

0.203 

0.222 

0.274 

Difficult 

One 

0.556 

0.447 

0.556 

0.229 

0.778 

0.438 

Three 

2.444 

0.594 

0.444 

0.247 

2.000 

0.521 

Female 

Easy 

One 

0.308 

0.238 

0.769 

0.195 

0.769 

0.174 

Three 

0.154 

0.100 

0.538 

0.169 

1.000 

0.228 

Difficult 

One 

1.846 

0.372 

0.538 

0.191 

2.923 

0.364 

Three 

1.538 

0.494 

0.692 

0.206 

3.308 

0.433 
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In  addition  to  the  three-way  interaction,  a  two-way  interaction  for  the  Gender  x  Primary 
Task  Difficulty  MANOVA  emerged,  F  (15,  6)  =  3.97,  X  =  0.092,  p  <  .05.  This  interaction  was 
also  subsequently  parsed  via  univariate  ANOVAs  for  each  dependent  variable.  Only  Total 


Enemies  to  Enter  Yellow  Zone  was  significant,  F  ( 1 ,  20)  =  5.62,  p  <  .05. 


Table  3.2.  Gender  x  Primary  Task  Difficulty 


Enter  Yellow 

Gender 

Primary  Task 

Mean 

SE 

Male 

Easy 

0.222 

0.199 

Difficult 

1.389 

0.421 

Female 

Easy 

0.885 

0.166 

Difficult 

3.115 

0.351 

A  main  effect  for  Primary  Task  Difficulty  was  also  observed,  F  (15,  6)  =  316.163,  X  = 
0.001,;?  <  .001.  With  the  exception  of  the  Number  of  Attacks  in  the  Red  Zone,  Enemy  Time  in  the 
Red  Zone,  Correct  Refuels,  Incorrect  Refuels,  all  other  univariate  tests  were  significant 
confirming  that  manipulation  of  primary  task  difficulty  was  successful  (see  Table  3.2).  Although, 
it  should  also  be  noted  that  some  dependent  variables  are  loaded  with  both  the  manipulation  of 
task  difficulty  as  well  as  participant  perfonnance.  For  example,  the  Number  of  Enemies 
Destroyed  is  highly  significant  between  the  two  task  conditions  because  manipulation  of  task 
difficulty  was  obtained  by  increasing  the  number  of  enemy  assets  that  the  participant  had  to 
engage.  This  information  is  conflated  with  perfonnance  data,  therefore,  making  it  difficult  to 
parse  which  variance  is  solely  a  function  of  the  task  manipulation  and  which  indicates  how  a 
participant  performed  under  the  different  task  constraints. 
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Table  3.3.  Univariate  tests  for  the  Main  Effect  Task  Difficulty 


Statistic 

Easy 

Condition 

Difficult 

Dependent  Measure 

Test 

F 

P 

Mean  SE 

Mean 

SE 

Enemies  Destroyed* 

F  (1,20)  =  322.198 

322.198 

0.000 

4.408 

0.278 

9.989 

0.329 

Attack  Green* 

F  (1,20)  =  108.207 

108.207 

0.000 

3.998 

0.366 

8.344 

0.494 

Attack  Yellow* 

F  (1,20)  =26.393 

26.393 

0.000 

0.391 

0.114 

1.795 

0.318 

Attack  Red 

F  (1,20)  =  0.215 

0.215 

0.648 

0.038 

0.032 

0.019 

0.023 

Fighters  Lost* 

F  (1,20)  =  35.289 

35.289 

0.000 

0.143 

0.097 

1.596 

0.260 

Enter  Green* 

F  (1,20)  =454.881 

454.881 

0.000 

5.250 

0.272 

11.444 

0.216 

Enter  Yellow* 

F  (1,20)  =  57.618 

57.618 

0.000 

0.553 

0.129 

2.252 

0.274 

Enter  Red 

F  (1,20)  =2.029 

2.029 

0.170 

0.066 

0.039 

0.229 

0.115 

Time  Green* 

F  (1,20)  =  10.356 

10.356 

0.004 

98.531 

5.630 

111.856 

3.620 

Time  Yellow* 

F  (1,20)  =  12.107 

12.107 

0.002 

26.726 

7.946 

51.870 

7.219 

Time  Red 

F  (1, 20)  =  1.751 

1.751 

0.201 

3.216 

1.893 

20.622 

12.894 

Correct  Refuels 

F  (1,20)  =  3.59 

3.590 

0.073 

1.573 

0.128 

1.276 

0.209 

Incorrect  Refuels 

F  (1, 20)  =  0.191 

0.191 

0.667 

0.075 

0.051 

0.103 

0.054 

Identify  To  Destroy 
Time* 

F  (1,20)  =22.072 

22.072 

0.000 

121.850 

8.818 

163.991 

7.537 

Aircraft  Lost  Fuel 

F  (1,20)  =  0.048 

0.048 

0.828 

0.521 

0.124 

0.558 

0.133 

The  multivariate  MANOVAs  for  the  Gender,  F  (15,  6)  =  1.261,  X  =  0.241,  and  Message 
main  effect,  F(15,  6)  =  1.397,  X  =  0.223,  were  not  significant.  Also,  the  Gender  x  Message  or  the 
Primary  Task  Difficulty  x  Message  interactions  were  also  not  significant,  F  (15,  6)  =  0.681,  X  = 
0.370  andF(15,  6)  =  0.832,  X  =  1.261,  respectively. 

3.2  Self-Report  Workload 

Self-reported  workload  assessed  using  the  NASA-TLX  was  evaluated  using  a  Gender 
(Male,  Female)  x  Primary  Task  Difficulty  (Easy,  Hard)  x  Message  Length  (1  Message,  3 
Messages)  with  repeated  measures  on  the  last  two  factors  MANOVA  that  included  each  of  the 
instruments  five  workload  subscales  (i.e.,  Mental,  Physical,  Temporal,  Performance,  Effort, 
Frustration).  Table  3.4  reports  the  means  and  standard  errors  for  each  of  the  observed  factos 
combinations.  No  higher  or  lower  order  interactions  were  identified  as  significant  for  self- 
reported  workload  on  the  TLX;  Gender  x  Primary  Task  Difficulty  x  Message  Length,  F  (6,  14)  = 
1.733,  X  =  0.568;  Gender  x  Primary  Task  Difficulty,  F  (6,  14)  =  0.506,  X  =  0.822;  Gender  x 
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Message  Length,  F  (6,  14)  =  0.868,  X  =  0.729.  However,  the  main  effect  for  Primary  Task 


Difficulty  was  significant,  F  (6,  14)  =  3.288,  X  =  0.415,  p  <  .05  providing  initial  evidence  that  the 


manipulation  was  successful  in  sufficiently  varying  workload  between  the  two  tasks  when 


collapsed  across  the  Message  Length.  A  strong  linear  trend  for  the  Message  Length  main  effect 
was  also  observed,  F  (6,  14)  =  2.433,  X  =  0.490,  p  =  .08. 

Table  3.4.  Means  and  standard  errors  for  the  weighted  NASA-TLX  dimensions  and  factor 
combination. 


Mental 

Temporal 

Performance 

Effort 

Frustration 

Gender 

Primary 

Task 

Message 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Male 

Easy 

One 

1.69 

0.43 

1.37 

1.16 

0.38 

1.12 

0.37 

Three 

1.62 

0.40 

1.03 

0.41 

0.78 

0.25 

1.30 

0.38 

0.58 

0.42 

Difficult 

One 

1.66 

0.42 

1.85 

0.41 

0.95 

0.33 

1.24 

0.32 

0.52 

0.52 

Three 

2.32 

0.53 

2.12 

0.42 

0.84 

0.28 

1.79 

0.34 

1.10 

0.48 

Female 

Easy 

One 

2.11 

0.37 

1.74 

0.36 

0.86 

0.33 

1.32 

0.28 

0.56 

0.32 

Three 

2.08 

0.35 

1.57 

0.35 

0.96 

0.22 

1.61 

0.33 

0.75 

0.36 

Difficult 

One 

2.44 

0.36 

2.32 

0.35 

1.04 

0.29 

1.24 

0.28 

1.59 

0.45 

Three 

2.91 

0.46 

2.03 

0.37 

1.09 

0.24 

1.81 

0.30 

1.03 

0.41 

*  Note  that  the  Physical  Workload  dimension  has  been  removed  for  presentation  and  that  values 
represent  weighted  TLX  scores  /  100. 


The  univariate  ANOVAs  were  subsequently  considered  to  evaluate  which  of  the  TLX 


subscales  contributed  to  either  of  the  multivariate  main  effects.  For  Primary  Task  Difficulty,  both 


the  Mental  and  Temporal  workload  dimensions  were  significant,  F ( 1,  19)  =  7.231  ,p  <  .02  and F 


(1,  19)  =  6.761,  p  <  .02,  respectively.  Thus,  mental  workload  significantly  increased  from  (M  = 


1.873,  SE  =  0.26)  for  the  Easy  Primary  Task  condition  to  (M  =  2.333,  SE  =  0.304)  for  the 


Difficult  Primary  Task  condition  irrespective  of  Message  Length.  Likewise,  Temporal  workload 


significantly  increased  from  (M  =  1.428,  SE  =  0.266)  for  the  Easy  Primary  Task  condition  to  (M 


=  2.078,  SE  =  0.260)  for  the  Difficult  Primary  Task  condition.  These  data  confirm  that  the 


primary  task  workload  manipulation  was  partly  successful. 
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3.3  Secondary  Task  Performance 


Separate  general  linear  models  were  used  to  detennine  differences  for  the  time  elapsed  to 
acknowledge  the  message  alert  ( Acknowledgement  Time)  and  the  time  to  send  the  final  message 
{Send  Tme ).  A  Gender  (Male,  Female)  x  Primary  Task  Difficulty  (Easy,  Hard)  x  Message  Length 
(1  Message,  3  Messages)  ANOVA  with  repeated  measures  on  the  last  two  factors  was  conducted 
for  both  dependent  variables.  Table  3.5  presents  means  and  standard  errors  for  the  three-way 
factor  combinations  for  both  dependent  variables.  The  Gender  x  Primary  Task  Difficulty  x 
Message  Length  for  message  Acknowledgment  Time  or  message  Send  Time  were  not 
significant,  F  (1,  17)  =  1.168  and  F  (1,  17)  =  0.681,  respectively.  The  following  lower  order 
interactions  for  message  Send  Time  were  also  not  significant:  Task  Difficulty  x  Message  Length, 
F  (1,  17)  =  2.049;  Gender  x  Message  Length,  F  (1,  17)  =  0.387.  While  the  Task  Difficulty  x 
Message  Length  interaction  was  also  not  significant  for  message  Acknowledgement  Time,  F  (1, 
17)  =  0.224,  the  Gender  x  Message  Length  interaction  did  emerge  as  significant,  F  (1,  17)  = 
4.09,  p  <  .05.  Post  hoc  t-tests  using  Bonferonni  corrections  confirmed  that  females  (M  = 
32034.23  ms,  SE  =  7008.87)  took  longer  to  complete  and  send  the  message  following  their 
acknowledgement  that  a  message  was  available  as  compared  to  males  (M  =  23934.73,  SE  = 
8218.66)  for  the  Three  message  condition. 


Table  3.5.  Means  and  standard  errors  for  the  secondary  task  dependent  variables. 


Gender 

Primary  Task 

Message 

Send  Time 

Acknowledge  Time 

Mean 

SE 

Mean 

SE 

Male 

Easy 

One 

6961.16 

733.96 

5726.30 

1536.52 

Three 

19916.76 

1841.27 

6047.86 

1820.72 

Difficult 

One 

8887.34 

1677.93 

6116.52 

7956.63 

Three 

27952.70 

16222.03 

9371.90 

4477.45 

Female 

Easy 

One 

9654.89 

625.92 

7110.27 

1310.35 

Three 

20371.96 

1570.24 

7087.91 

1552.71 

Difficult 

One 

10233.84 

1430.94 

26523.46 

6785.44 

Three 

43696.51 

13834.19 

18992.84 

3818.38 
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Two  additional  main  effects  were  further  identified  as  significant.  First,  a  main  effect  for 
Send  Time  was  observed  for  Message,  F  (1,  17)  =  15.215,  p  <  .001,  indicating  participants 
expectedly  required  more  time  to  enter  longer  messages.  On  average  participants  required 
8934.306  ms  (SE  =  702.963)  to  enter  one  message  and  27984.482  ms  (SE  =  5400.717)  to  enter 
three  messages.  Second,  Acknowledgement  Time  was  observed  to  significantly  vary  with 
Primary  Task  Difficulty,  F  (1,  17)  =  6.584,  p  <  .05.  That  is,  participants  were  significantly 
slower  to  respond  to  the  occurrence  of  a  message  alert  during  the  Easy  (M  =  6493.083,  SE  = 
862.763)  compared  to  alerts  occurring  during  the  Difficult  condition  (M  =  15251.181,  SE  = 
3703.183).  On  the  other  hand,  the  average  time  participants  required  to  send  the  message  did  not 
vary  as  a  function  of  Primary  Task  Difficulty,  F  (1,  17)  =  2.252. 

3.4  Heart  Rate  Variability 

Separate  general  linear  models  were  used  to  determine  differences  for  the  Very  Low, 
Low,  High,  and  Very  High  heart  rate  variability  frequency  components.  A  Gender  (Male, 
Female)  x  Primary  Task  Difficulty  (Easy,  Hard)  x  Message  Length  (1  Message,  3  Messages) 
ANOVA  with  repeated  measures  on  the  last  two  factors  was  conducted  for  each  dependent 
variable.  The  omnibus  interaction  for  the  Very  Low,  and  High  Frequency  bands  were  not 
significant,  F  (1,  21)  =  0.710  and  F  (1,  21)  =  1.436,  respectively.  A  strong  linear  trend  was 
observed  for  the  Very  High  HRV  frequency  band,  F  (1,  21)  =  3.191,  p  =  .08.  Further,  the  three- 
way  interaction  for  the  Low  frequency  band  was  significant,  F  ( 1 ,  21)  =  4.667,  p<  .05. 

Table  3.6  presents  the  means  and  standard  errors  for  the  interaction  factor  combinations.  Select 
follow-up  analyses  indicated  a  trend  between  greater  HRV  in  the  Easy/One  Message  and 
Easy/Three  Message  condition  compared  to  each  of  the  other  experimental  conditions.  However, 


32 


follow-up  comparisons  did  not  reveal  any  significant  differences  within  the  larger  significant 
three  -way  interaction. 


Table  3. 6.  Means  and  standard  errors  for  the  heart  rate  variability  dependent  variables  by  factor 
combination. 


Gender 

Primary 

Task 

Message 

VLFB 

LFB 

HFB 

VHFB 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Male 

Easy 

One 

25.743 

4.025 

12.117 

4.255 

10.592 

3.664 

4.711 

1.520 

Three 

33.240 

6.921 

18.374 

6.053 

9.965 

2.948 

6.624 

2.284 

Difficult 

One 

20.054 

2.248 

6.395 

1.124 

5.275 

1.286 

5.249 

1.801 

Three 

21.659 

2.900 

4.304 

0.617 

2.105 

0.434 

1.237 

0.371 

Female 

Easy 

One 

22.023 

3.530 

8.382 

3.732 

3.598 

3.214 

0.766 

1.333 

Three 

20.795 

6.071 

5.533 

5.308 

2.882 

2.586 

1.067 

2.003 

Difficult 

One 

20.519 

1.972 

4.313 

0.986 

1.945 

1.128 

0.926 

1.579 

Three 

19.829 

2.543 

4.331 

0.542 

1.969 

0.381 

1.092 

0.325 

Note:  Values  reported  refer  to  power  spectral  density  in  sec2/Hz. 


In  addition  to  the  significant  interaction,  a  main  effect  for  Primary  Task  Difficulty  was 


also  observed  for  both  the  Low  and  High  Frequency  band  components,  F(l,  21)  =  4. 170, /?  <  .05 


and  F  ( 1,21)  =  5.663,  p  <  .05.  Participants  occasioned  significantly  less  variability  for  both  of 


these  spectral  bands  in  the  Difficult  as  compared  to  the  Easy  primary  task  condition  (Low:  M  = 


4.836,  SE  =  0.413  versus  M=  11.102,  SE  =  3.198;  M=  2.824,  SE  =  0.505  versus  High:  M  = 
6.759,  SE  =  2.091).  The  main  effect  for  the  Very  Low,  F  (1,  21)  =  2.489,  and  the  Very  High,  F 
(1,  21)  =  0.1 18,  frequency  components  were  not  significant. 
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HRV  Low  Frequency  Power  Spectral  Density 
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Figure  3. 1.  Heart  rate  variability  power  spectral  density  for  the  low  frequency  range 


HRV  High  Frequency  Power  Spectral  Density 


Male  Easy  Female  Easy  Male  Difficult  Female  Difficult 


Figure  3.2.  Heart  rate  variability  power  spectral  density  for  the  high  frequency  range. 
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3.4.1  Event-Related  Potentials 


Separate  general  linear  models  were  used  to  determine  differences  for  the  P3  Amplitude, 
P3  Latency,  N1  Amplitude,  and  N1  Latency.  A  Gender  (Male,  Female)  x  Primary  Task  Difficulty 
(Easy,  Hard)  x  Message  Length  (1  Message,  3  Messages)  ANOVA  with  repeated  measures  on 
the  last  two  factors  was  conducted  for  each  dependent  variable.  A  significant  three-way 
interaction  for  P3  Latency  emerged,  F(l,  8)  =  5.214 ,p  <  .05.  However,  follow-up  analyses  were 
inconclusive  because  of  the  low  sample  size  in  each  of  the  factor  cells.  The  omnibus  three-way 
interactions  for  P3  Amplitude,  N1  Latency,  N1  Amplitude  were  not  significant,  F  { 1,  8)  =  0.642, 
F  (1,  8)  =  0.216,  F  (1,  8)  =  0.216.  Other  lower  order  interaction  and  main  effects  were  also  not 


significant. 


Table  3.7.  Means  and  standard  errors  for  electrocortical  dependent  variables  by  factor 
combination. 


Gender 

Primary 

Task 

Message 

P3  Amplitude 

N1  Amplitude 

P3  Latency 

N1  Latency 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Male 

Easy 

One 

0.556 

0.265 

-0.896 

0.184 

284.200 

11.218 

133.000 

6.631 

Three 

0.972 

0.408 

-0.593 

0.312 

319.800 

15.365 

124.000 

5.990 

Difficult 

One 

0.896 

0.336 

-0.724 

0.429 

314.400 

14.025 

120.400 

3.593 

Three 

0.446 

0.193 

-0.794 

0.170 

286.600 

8.149 

114.000 

5.534 

Female 

Easy 

One 

0.890 

0.265 

-1.019 

0.184 

322.200 

11.216 

132.800 

6.631 

Three 

1.058 

0.408 

-1.332 

0.312 

328.200 

15.365 

120.600 

5.990 

Difficult 

One 

0.816 

0.336 

-1.278 

0.429 

282.200 

14.025 

127.600 

3.593 

Three 

0.602 

0.193 

-1.630 

0.170 

314.800 

8.149 

115.400 

5.534 
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Grand  Averaged  ERP 


Figure  3.3.  Grand  averaged  ERP  for  ad  participants  and  cortical  channels  (P3,  Pz,  P4)  and 
shown  by  Task  and  Messages  factor  combinations. 

4.  Discussion 

The  purpose  of  this  experiment  was  to  examine  operator  workload  demands  when  a 
messaging  task  is  introduced  as  a  secondary  task  requirement.  The  messaging  task  in  this 
experiment  required  participants  to  replicate  directional  coordinate  segments  (i.e.,  26  E,  120  N) 
in  either  single  or  triple  message  chunks  at  various  intervals  during  a  synthetic  command  and 
control  primary  task. 

Generally  speaking,  participants  exhibited  both  increased  Mental  and  Temporal  self- 
reported  irrespective  of  Message  Length,  suggesting  that  the  primary  task  workload  manipulation 


was  partially  successful.  A  strong  linear  trend  was  also  observed  for  manipulations  of  Message 
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Length,  however  this  effect  was  not  significant.  The  secondary  task  manipulation  was 
furthermore  revealing,  with  females  taking  on  average  longer  to  complete  and  send  the  message 
following  their  acknowledgement  that  a  message  was  available  as  compared  to  males  for  the 
Three  message  condition.  Participants  were  also  significantly  slower  when  responding  to  a 
message  alert  during  given  increased  primary  task  difficulty.  On  the  other  hand,  the  average  time 
participants  required  to  send  the  message  did  not  vary  as  a  function  of  Primary  Task  Difficulty 
suggesting  that  participants  tended  not  to  time  share  between  the  primary  and  secondary  task 
once  the  message  entry  began,  but  did  adopt  a  hierarchical  coping  strategy  by  choosing  only 
acknowledging  the  message  alert  when  afforded  by  a  event  gap  in  the  primary  task.  Participants 
were  also  observed  to  occasion  significantly  less  variability  for  both  of  the  Low  and  High 
frequency  spectral  bands  in  the  Difficult  as  compared  to  the  Easy  primary  task.  However,  follow¬ 
up  analyses  of  these  results  were  inconclusive.  Despite  a  significant  finding  for  P3  latency, 
findings  related  to  ERP’s  were  equally  as  inconclusive  as  those  observed  with  HRV.  That  is,  a 
distinct  pattern  of  results  failed  to  emerge  given  present  task  conditions.  On  the  one  hand,  while 
physiological  sensitivity  is  a  major  reason  to  include  measures  of  psychophysiological  function, 
it  is  conceivable  that  varying  levels  of  expertise  played  a  heavy  role  in  obscuring  a  clear  and 
unified  physiological  profile. 

There  are  a  number  of  limitations  of  the  current  study  that  should  be  addressed.  First,  as 
with  many  studies,  a  sample  of  convenience  was  used  because  ethnographic  or  other  more 
ecological  based  evaluations  where  not  possible.  One  then  must  question  how  the  abilities  of  the 
operators  in  this  study  influenced  the  observed  pattern  of  results.  Moreover,  the  major  question  is 
how  well  these  results  describe  how  experts  might  react  when  confronted  with  the  real  world 
analog  of  this  task?  This  challenge  however  is  not  a  native  problem  to  this  study,  but  rather 
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represents  a  larger  issue  where  samples  of  convenience  are  used  in  lieu  of  competent  operators. 
Arguably,  the  various  task  requirements  in  this  study  were  equivalent  given  the  samples  level  of 
expertise  with  the  experimental  task  relative  to  the  task  requirements  to  a  real  world  operator. 
However,  this  is  an  assumption  that  is  made  for  the  purposes  of  this  kind  of  research  and  one 
must  be  careful  to  consider  the  obtained  results  within  this  context. 

In  addition  to  the  studies  sample,  some  might  argue  that  laboratory  evaluation  of  mental 
workload  does  not  permit  robust  assessment  of  true  operator  conditions  in  the  real  world.  On  the 
one  hand,  the  experimental  capacity  to  instantiate  the  substrata  of  real  world  stressors  into  a 
laboratory  task  are  indeed  challenging.  For  example,  it  is  self-evident  that  replication  of  the  task 
conditions  during  the  first  Gulf  War  to  mimic  the  level  of  war  fighter  stress  induced  would  be 
impractical.  On  the  other  hand,  this  study  clearly  observed  task  engagement  and  level  of  self- 
reported  and  physiological  workload  across  task  conditions.  Therefore,  it  is  reasonable  to 
conclude  that  participant  engagement  in  this  study  reflected  a  true  snapshot  of  what  may  be 
experienced  by  more  experienced  operators,  particularly  where  another  novel  task  requirement 
(i.e.,  chat)  is  introduced  into  one’s  primary  scope  of  responsibility. 

During  the  Gulf  War,  as  with  modern  chat  applications,  the  lack  of  instant  messaging 
demand  control  is  a  major  problem  that  is  not  easily  rectified.  In  normal  voice  communication, 
for  example,  it  is  common  for  one  member,  even  when  engaging  in  a  long  series  of  statements  to 
wait  or  prompt  for  acknowledgement  from  the  other  conversation  member.  Silence  for  too  long 
of  period,  in  this  particular  case,  may  denote  that  the  other  party  is  not  receiving  or  fully  attentive 
of  the  incoming  information.  While  there  are  exceptions,  chat  applications  often  are  interacted 
with  as  tertiary  to  some  primary  task.  If  primary  task  requirements  become  to  high,  performance 
in  the  secondary  task  will  decline.  So,  how  then  does  one  determine  the  other  member’s  demand 
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without  adding  additional  workload?  What  happens  when  a  message  is  critical?  How  long  do 
you  wait  for  an  acknowledgement  before  concluding  that  your  message  has  not  been  conveyed? 
If  a  single  operator  is  handling  chat  communication  from  multiple  sources,  how  are  such 
messages  prioritized  such  that  the  operator  can  efficiently  act  on  requests  accordingly? 

The  standardization  of  a  military  specific  chat  grammar  is  another  important 
consideration  for  fully  realizing  this  technology.  Many  researchers  have  examined  the  emergence 
of  localized  chat  grammars  that  evolve  within  user  social  networking  communities.  Laugh  out 
loud  “lol”  is  a  common  example,  (see  http://en.wikipedia.org/wiki/List_of_Internet_slang).  The 
notion  of  chat  grammar  is  to  accelerate  information  exchange  from  the  cumbersome  act  of 
keyboard  entry.  However,  if  chat  is  going  to  be  used  as  a  means  of  formal  military 
communication,  it  is  critical  that  each  chat  member  clearly  understand  the  infonnation  quickly 
and  unambiguously.  This  is  particularly  important  where  multiple  operators  might  perform  the 
same  job  in  different  shifts  but  the  shift  operator  needs  to  understand  both  the  context  and 
meaning  of  earlier  exchanges.  However,  there  are  several  strategies  that  may  mitigate  this 
problem  such  as  chat  certification  for  domain  specific  tokens  (i.e.,  keywords)  that  operators 
would  use  within  certain  job  functions. 
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Introduction 


This  study  addresses  the  need  to  better  understand  operator  workload/performance  issues 
in  Battle  Management  Command  and  Control  (BMC2)  domains.  The  project’s  primary  aim  will 
be  to  explicate  workload/perfonnance  dissociations  by  mapping  data  derived  from  existing  C2 
Simulated  Training  Environments  (STE;  e.g.,  Distributed  Dynamic  Decision-Making;  DDD) 
with  a  multi-modal  assessment  protocol  that  will  bridge  both  direct  and  indirect  forms  of  inquiry. 
In  this  analysis,  the  issue  of  individual  operator  workload  associated  with  the  integration  of 
collaborative  technologies  (CT)  into  workflow  operations  will  be  of primary  interest. 

A  quintessential  component  of  command  and  control  environments  are  the  unique  and 
dynamic  contingencies  imposed  upon  the  operator.  Command  and  control  (C2)  operators  are 
often  required  to  make  rapid  and  important  decisions  in  situations  of  scant  and  overwhelming 
intelligence.  Thus,  tools  that  can  aid  orchestration  and  management  of  information,  or  by  the 
facilitation  of  infonnation  exchange  via  team  coherence,  have  the  potential  to  significantly 
improve  mission  effectiveness.  However,  it  is  also  possible  that  such  interventions  will  increase 
the  workload  on  C2  operators  and  thereby  impair  performance  in  stressful  operational  conditions. 
Analysis  of  the  relation  between  performance  and  workload  with  respect  to  addition  of  new 
technologies  is  therefore  critical  for  ensuring  mission  success. 

Collaborative  technologies  intend  to  enhance  operator  and  team  efficiency  by  serving  as  a 
conduit  for  users  to  better  share,  organize,  and  process  information.  Integration  of  network-based 
tools  such  as  chat,  whiteboard,  photo-  and  application-  sharing,  video-  and  audio-  conferencing, 
and  seamless  file  transfer  capabilities,  to  cite  a  few  examples,  have  the  potential  to  significantly 
improve  operator  productivity,  decision-making,  problem-solving,  and  team  situational 
awareness.  Though  novel  CT  are  meant  to  replace  outdated  systems  for  information  exchange, 
improve  operator  efficiency,  or  to  simplify  complex  collaborative  tasks,  there  are  important 
questions  to  answer  regarding  the  performance  and  workload  effects  of  integrating  network- 
based  information  sharing  tools  into  contemporary  BMC2  operations. 

To  date,  research  evaluating  the  efficacy  of  various  CT  has  indicated  limited  effectiveness 
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of  these  technologies  for  enhancing  performance.  For  example,  Navy  operational  commanders 
during  Operation  Iraqi  Freedom  reported  that  the  amount  of  information  generated  by  instant 
messaging  (IM)  became  problematic  because  it  became  difficult  to  extract  and  synthesize 
(Caterinicchia,  2003).  In  a  recent  study,  Cummings  (2005)  investigated  the  use  of  IM  capabilities 
in  a  tactical  Tomahawk  simulation,  finding  that  participants  would  persistently  fixate  on  the  IM 
window  despite  being  instructed  that  the  responding  to  message  events  was  secondary  to  a 
primary  task  of  monitoring  and  responding  to  missile  retargeting.  While  CT  will  certainly  have 
their  place  in  network-centric  warfare,  it  is  crucial  that  these  technologies  be  systematically 
evaluated  for  their  impact  on  performance  and  workload. 

The  human  brain  has  evolved  in  conditions  where  processing  constraints  have 
necessitated  neural  systems  that  seek  to  reduce  the  burden  of  a  data-rich  environment  by  filtering 
information  through  selective  attention.  The  brain  must  process  and  organize  the  voluminous 
data  collected  through  the  sensory  systems  into  a  meaningful  information  array  that  affords 
coordination  of  timely  response  to  novel  stimulus  contingencies  appearing  unpredictably  in 
space  and  asynchronously  in  time.  However,  the  systems  that  process  this  information  are 
inherently  limited  in  their  capacities  (Broadbent,  1958).  These  capacity  limits  are  manifested  in 
diverse  environments  as  the  cognitive  workload  and  stress  associated  with  task  performance.  To 
date,  there  has  been  limited  theoretical  or  empirical  research  to  articulate  the  factors  that 
influence  the  complete  link  between  perfonnance  and  perceived  workload.  Given  that  different 
task  domains  produce  different  performance-workload  relations,  a  central  question  emerges: 
what  factors  drive  subjective  measures  of  workload  and  what  information  do  these  measures 
provide  that  are  not  intrinsic  to  performance  measures?  Such  relations  provide  only  on  single 
scalar  link  between  multiple  faces  of  performance. 

Method 

Participants 

Participants  will  be  selected  from  undergraduates  enrolled  in  psychology  courses  or  in  the 
UCF  community.  There  will  be  no  inclusion  or  exclusion  criteria  except  that  participants  must 
have  normal  or  corrected  to  normal  vision  and  have  no  known  hearing  impairments.  Participation 
will  be  completely  voluntary,  and  will  not  negatively  affect  students’  grades  in  any  way. 
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Students  who  choose  to  participate  in  this  research  effort  will  be  compensated  with  extra  course 
credit.  To  complete  the  study,  we  anticipate  two  one-hour  sessions.  The  first  session  will  consist 
mostly  for  training  participants  on  the  primary  command  and  control  task,  while  the  second 
session  will  be  dedicated  to  actual  testing.  Each  session  is  not  anticipated  to  exceed  one  hour  and 
thirty  minutes.  The  experiment  will  not  begin  until  the  participant  has  read  and  signed  the 
informed  consent  (See  Appendix  A)  and  had  any  questions  or  concerns  addressed.  All  individual 
information  gathered  during  this  study  will  be  kept  strictly  confidential.  Further,  the  information 
provided  throughout  participation  in  this  study  will  be  stored  in  such  a  way  that  it  will  not  be 
connectable  to  people’s  names.  Further,  experimental  data  will  be  stored  in  such  a  way  that  there 
will  be  no  way  of  linking  people  to  data  file(s),  thus  ensuring  privacy.  Informed  consent  fonns 
will  they  be  kept  in  a  locked  filing  cabinet  in  a  locked  office  AND  separate  from  the  data. 

Experimental  Design 

The  primary  task  in  each  experiment  will  require  participants  to  engage  in  a  synthetic 
task  environment  that  mimics  many  of  the  characteristics  common  to  command  and  control 
domains.  The  simulation  software  is  called  the  Distributed  Dynamic  Decision-Making  by 
Aptima.  The  software  allows  one  to  control  icons  which  represent  various  battlefield  assets  such 
as  tanks,  planes,  bases,  etc.  the  goal  of  the  software  is  for  the  participant  to  make  strategic 
decisions  such  that  they  fulfill  mission  objectives  (see  Figure  1). 
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Figure  1.  Primary  Task  Screenshot 


The  specific  mission,  in  the  proposed  experiment,  will  require  participants  protect  a  base  against 
attacks  from  enemy  forces.  A  secondary  task  will  require  that  participants  respond  to  a  tone  that 
will  occur  at  intermittent  points  during  the  primary  task.  Participants  will  be  instructed  to  click  a 
small  button  located  on  the  screen  when  they  hear  this  tone.  Once  a  button  is  pressed,  a  chat 
window  appears  with  latitude  and  longitude  coordinates  that  the  participant  will  be  asked  to 
replicate  as  quickly  and  as  accurately  as  possible  without  compromising  their  responsibilities  on 
the  primary  task.  This  paradigm,  therefore,  will  allow  us  to  assess  the  impact  of  instant 
messaging  (collaborative  technology)  during  a  primary  C2  task.  We  will  assess  behavioral  data 
with  respect  to  the  response  latency  to  press  the  button  following  the  tone  and  to  replicate  the 
coordinates  in  addition  to  any  input  errors  that  may  occur  as  the  primary  and  secondary  task 
collectively  increase  participant  workload.  During  the  experiment  we  will  also  monitor  eye  point 
of  gaze  data.  This  behavioral  measure  of  attentional  allocation  will  be  helpful  in  detennining 
how  attention  is  allocated  between  the  primary  and  secondary  tasks.  In  addition  to  these 
behavioral  measures,  physiological  markers  of  workload  will  also  be  collected  in  the  form  of 
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electrocortical  potentials  and  electrocardiographic  activity.  This  will  require  that  participants 
wear  small  electrodes  during  the  experiment.  In  addition  to  this  instrumentation,  the  following 
questionnaires  will  also  be  administered: 

Measures  of  Workload  and  Mood  -  Perceived  mental  workload  will  been  measured  by  the 
NASA  Task  Load  Index  (NASA-TLX;  Hart  &  Staveland,  1988),  a  well  regarded  instrument  that 
gauges  six  dimensions  of  perceived  workload  on  a  scale  of  0  to  100.  In  all  experiments  mood 
states  will  be  measured  using  the  Dundee  State  Questionnaire  (DSQ),  a  multidimensional 
instrument  for  assessing  transient  states  associated  with  mood,  arousal  and  fatigue  (Matthews  et 
ah,  1999;  see  Appendix  C  for  pre-  and  post-versions  of  this  questionnaire).  This  instrument  has 
been  shown  to  be  sensitive  in  investigating  the  effects  of  task  factors  on  operator  mood  state 
(Grier,  et  al.,  2004;  Helton,  Dember,  Warm,  &  Matthews,  1999;  Matthews,  et  al.,  1999;  Szalma 
et  al.,  2004). 

Personality  Measures  -  Trait  and  state  anxiety  will  be  measured  using  the  State-Trait  Anxiety 
Inventory  (STAI;  Spielberger,  Gorsuch,  Lushene,  Vagg,  &  Jacobs,  1983).  The  STAI  assess  trait 
feelings  of  emotionality  as  well  as  transient  feelings  of  emotionality  experienced  during  task 
performance.  The  Eysenck  Personality  Inventory  (EPI)  will  also  be  administered.  The  EPI 
produces  to  personality  scores  of  neurotiscm  and  introversion. 


Description  of  Experimental  Protocol 

After  a  brief  verbal  description  of  the  experiment  by  the  experimenter,  participants  will 
be  presented  with  an  informed  consent  form  that  explains  their  rights  as  participants,  including 
the  right  to  withdraw  their  participation  at  any  time  without  penalty  (See  Appendix  A  for  a  copy 
of  the  informed  consent  form).  The  experiment  orientation  session  will  not  begin  until  the 
participant  has  read,  had  any  questions  answered,  and  signed  the  informed  consent  form.  All 
data  sheets  will  be  coded  with  a  participant  number  to  maintain  confidentiality. 

Following  informed  consent,  participants  will  be  asked  to  complete  several  of  the  above 
questionnaires  followed  by  a  presentation  that  will  detail  instructions  for  how  to  use  the  primary 
task.  At  the  first  session,  any  questions  regarding  the  task  objectives  as  well  as  the  task  itself  will 
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be  addressed.  Once  participants  have  completed  this  initial  session  they  will  be  scheduled  for  an 
experimental  session. 

Once  the  participant  arrives  for  the  experimental  session  they  will  be  greeted  and  then 
asked  to  play  a  practice  session  for  the  primary  task,  and  any  questions  about  how  to  use  the  task 
will  be  answered.  Following  this  practice  task  (~  15  minutes)  the  physiological  instrumentation 
will  be  affixed  and  the  eye  tracker  calibrated  (~15  minutes).  The  participant  will  then  engage  in 
four  task  scenarios  each  approximately  lasting  10  minutes.  Upon  completion  of  the  task, 
participants  will  complete  a  computerized  version  of  the  NASA-TLX  while  the  next  scenario  is 
loaded.  A  de-briefing  form  will  be  given  to  the  participant  at  the  end  of  the  session. 

Potential  Risks  and  Benefits 

The  display  used  for  the  tasks  in  the  proposed  study  is  similar  to  graphic  displays  used  in 
commercially  available  computer  programs,  which  have  no  known  negative  long-term  effects  on 
participants.  Application  of  the  sensors  to  the  body  poses  minimal  problems.  A  sensor  is  applied 
1)  by  first  cleaning  the  skin  by  rubbing  briskly  with  a  tissue,  2)  applying  a  jelly,  and  3)  attaching 
the  sensors  to  the  skin  with  a  tape-like  collar.  Individuals  with  high  sensitivity  to  cosmetics  or 
creams,  or  extreme  skin  allergies  may  find  that  a  short-lived  redness  at  certain  sites  develops 
after  the  sensors  are  applied,  although  most  people  find  the  procedure  harmless.  The  possibility 
of  infection  at  any  site  where  sensors  are  applied  is  non-existent  in  most  cases,  and  minimal  at 
most,  as  proper  cleaning  and  sterilization  procedures  are  implemented  at  all  times. 
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Informed  Consent 


The  purpose  of  this  study  is  to  investigate  variables  that  influence  how  people  perform 
when  using  collaborative  technologies  in  command  and  control  environments. 

Anonymity: 

All  individual  information  gathered  during  this  study  will  be  kept  strictly  confidential. 
Further,  the  information  provided  throughout  participation  in  this  study  will  be  stored  in  such  a 
way  that  it  will  not  be  connectable  to  people’s  names.  Further,  experimental  data  will  be  stored 
in  such  a  way  that  there  will  be  no  way  of  linking  people  to  data  file(s),  thus  ensuring  privacy. 

What  is  being  asked: 

Participation  will  principally  consist  of  filling  out  demographic  and  survey  information 
forms  and  then  performing  PC-based  tasks  in  another  session.  The  tasks  you  perform  on  the 
computer  will  involve  a  game-like  task  that  will  require  you  to  control  battlefield  assets  and 
counteract  attacks  against  those  assets  by  enemy  forces.  During  this  primary  task,  you  will  be 
asked  to  respond  to  instant  messages  when  prompted.  The  experimenter  will  explain  how  to  do 
each  task  and  the  nature  of  the  secondary  task.  After  finishing  each  task,  you  will  be  asked  to  fill 
out  a  few  questionnaires  about  your  experiences  during  the  session.  The  experiment  will  take 
approximately  1  hour  and  thirty  minutes  to  complete,  and  will  require  you  to  return  to  the 
laboratory  on  one  other  occasion  for  approximately  the  same  amount  of  time. 

Other  concerns: 

The  researchers  agree  to  answer  any  questions  that  you  may  have  at  this  time  or  at  any 
time  during  the  duration  of  the  study.  You  do  not  have  to  answer  any  question  that  you  do  not 
wish  to  answer.  If  at  anytime  during  the  study  you  feel  uncomfortable  in  any  way,  you  can  and 
should  inform  the  researcher  and  the  study  will  be  terminated  immediately  with  no  penalty  or 
loss  of  benefit. 

Previous  experience  with  computer-based  tasks  has  shown  that  some  participants  may 
experience  some  level  of  physical  discomfort,  however,  there  is  no  major  health  risk  involved  as 
a  result  of  participation  in  this  study.  However,  if  at  any  time  you  feel  disoriented,  sick,  or 
nauseated,  inform  the  researcher  and  the  study  will  be  terminated  immediately  without  penalty  or 
loss  of  benefit.  Furthermore,  the  study  poses  no  known  psychological  risks.  Application  of  the 
sensors  to  the  body  poses  minimal  problems.  A  sensor  is  applied  1)  by  first  cleaning  the  skin  by 
rubbing  briskly  with  a  tissue,  2)  applying  a  jelly,  and  3)  attaching  the  sensors  to  the  skin  with  a 
tape-like  collar.  Individuals  with  high  sensitivity  to  cosmetics  or  creams,  or  extreme  skin 
allergies  may  find  that  a  short-lived  redness  at  certain  sites  develops  after  the  sensors  are  applied, 
although  most  people  find  the  procedure  harmless.  The  possibility  of  infection  at  any  site  where 
sensors  are  applied  is  non-existent  in  most  cases,  and  minimal  at  most,  as  proper  cleaning  and 
sterilization  procedures  are  implemented  at  all  times. 

Participation  is  voluntary  and  you  may  terminate  participation  in  the  study  at  any  time 
with  no  penalties  by  the  researchers  or  the  University  and  that  the  only  benefit  to  you  for 
participation  is  the  payment  you  will  receive  for  your  participation.  Compensation  for  your 
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participation  will  be  extra  course  credit  provided  via  Experimentrak.  You  will  receive  1  credit 
point  for  each  half  hour  of  participation.  If  you  only  complete  part  of  the  experiment,  you  will 
receive  compensation  for  the  amount  time  you  have  spent  in  the  experiment  rounded  to  the 
nearest  half-hour  interval. 

If  you  believe  you  have  been  injured  during  participation  in  this  research  project,  you  may  file  a 
claim  with  UCF  Environmental  Health  &  Safety,  Risk  and  Insurance  Office,  P.O.  Box  163500,  Orlando, 
FL  32816-3500  (407)  823-6300.  The  University  of  Central  Florida  is  an  agency  of  the  State  of  Florida 
for  purposes  of  sovereign  immunity  and  the  university's  and  the  state's  liability  for  personal  injury  or 
property  damage  is  extremely  limited  under  Florida  law.  Accordingly,  the  university's  and  the  state's 
ability  to  compensate  you  for  any  personal  injury  or  property  damage  suffered  during  this  research 
project  is  very  limited. 

Information  regarding  your  rights  as  a  research  volunteer  may  be  obtained  from: 

IRB  Coordinator 
Institutional  Review  Board  (IRB) 

University  of  Central  Florida  (UCF) 

12201  Research  Parkway,  Suite  501 
Orlando,  Florida  32826-3246 
Telephone:  (407)  823-2901 

Questions  about  anything  having  to  do  with  this  study  can  be  addressed  to: 

Aaron  R.  Duley,  Ph.D. 

Institute  for  Simulation  and  Training 

University  of  Central  Florida 

3100  Technology  Parkway 

Room  333 

Orlando,  FU  32826 

Phone:  (407)-823-1492 

E-mail:  aaronduley@gmail.com 

I  have  read  the  procedure  described  above.  I  understand  all  points  and  agree  to 
participate  in  the  procedure  and  I  have  received  a  copy  of  this  description.  I  further  state 
and  certify  that  I  am  at  least  18  years  of  age. 


Signature  of  Participant 


Signature  of  Researcher 


Date 


Date 
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7.  Appendix  B 


Collaborative  Technologies  in  BMC2  Study 


Drs.  Aaron  R.  Duley,  James  Szalma,  P.  A.  Hancock 
Minds  in  Technology,  Machines  in  Thought  Laboratory 
Institute  for  Simulation  and  Training 
University  of  Central  Florida 


Research  sponsored  by  the  United  States  Air  Force  in  collaboration  with  the  Air  Force  Research 
Laboratory.  Special  thanks  to  Dr.  Todd  Nelson,  Robert  Bolia,  Brent  Miller,  Dr.  Benjamin  Knott 


Today 


•  Informed  consent 

•  Questionnaires 

•  About  the  experiment 

•  Schedule  a  time 
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Informed  Consent 


Please  take  a  few  minutes  to  read  and 
sign  the  informed  consent  form 

If  you  have  any  question  feel  free  to  ask 
the  experimenter 


Questionnaires 


•  Please  take  a  few  minutes  to  complete  the  questionnaires 

•  Do  not  put  your  name  on  any  of  the  questionnaires 

•  Please  ask  the  experimenter  if  you  have  any  questions 
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About  The  Experiment 


•  You  will  be  playing  the  role  of  a 
weapons  director  and  strike  operator  in 
a  command  and  control  scenario 

•  The  Tanker  Scenario  is  a  simulated  air 
battle  management  command  and 
control  task  developed  with  guidance 
from  subject  matter  experts  in 
conjunction  with  the  Air  Force 
Research  Laboratory 


•  Operators  communicate  and 
coordinate  to  intercept  and  attack 
enemy  targets,  rearm  and  refuel 
friendly  fighters  with  airborne  tankers. 


Testbed 


•  The  Dynamic  Distributed 
Decision-Making  Testbed  (DDD) 
by  Aptima,  Inc. 


•  You  own  and  control  color- 
coded  fighters,  tankers,  bases, 
etc. 


•  Coordinated  attack  and 
defense 


•  Protect  friendly  assets, 
eliminate  enemy  threats 
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Testbed  Example 


Using  DDD  -  Asset  Menu 


The  majority  of  commands  will 
be  issued  by  Right-Clicking  on 
a  given  asset 


Move 

Ortoil 


Refuel 

Move  to  Tanker 

Restock 

Pursue 


«  v 


jui 


A»^.k 

Return 


Info  on  Asset 
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Using  DDD  -  Asset  Information 


To  get  information  about  an 
asset: 

1.  Right-Click  on  the  asset 

2.  Select  ‘Info  on  Asset’  from 
the  Menu 


Move 

Ortoil 

Refuel 

.  Move  to  Tanker 
Restock 
Pursue 
Aty,k 
Return 

Info  on  Asset 


Using  DDD  -  Asset  Information  Dialog 
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Using  DDD  -  Orbiting  Assets 


f->ruv 


Move  to  Tanker 


F  III  HJ8 


'y 


Return 


info  on  Asset 


-Lr.r.- 


The  ‘Orbit’  command  will  place  a  moveable  asset  into 
a  stationary  position. 


Using  DDD  -  Rings 
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Using  DDD  -  Attacking 


To  attack  using  an  asset: 

1 .  Right-Click  on  the  asset 

2.  Select  ‘Attack’  from  the 
Menu 

3.  Left-Click  the  enemy  asset 
to  attack 


.'it  II 


•  •-!  If 


Move  to  Tanker 


Pure  i“ 


■y 


■  <a  in 


Info  on  Asset 


The  ‘Attack’  command  will  direct  you  asset  to  destroy 
an  enemy. 


Using  DDD  -  Attacking 


•  Attack  errors 


Error  on  attack 

|ox] 

1  Cannot  Attack  Target  AMiGZS-ZOS  at  (375.8,140.5)  with  Asset  lceT(F-18)  at  (312.1 ,76.1 ) 

•  ...Out  of  Attack  range. 

L  - 
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Using  DDD  -  Attack  Confirmation 


•  Attack  confirmation 


Using  DDD  -  Attacking 


\ 

Busy  Icons 

_ j 


> 

Task  completion 
indicator 

_ / 
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Using  DDD  -  Refueling 
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Using  DDD  -  Refueling 


Using  DDD  -  Launching  an  Aircraft 


66 


Your  function 


•  As  Weapons  Director  (WD),  you  will  run  air  defense  operations  by 
coordinating  your  assets  while  acting  on  incoming  information  from  you 
tactical  display 

•  WDs  have  global  picture  of  the  airspace  on  their  tactical  displays  that  is 
provided  by  an  Airborne  Warning  and  Control  System  (AWACS) 


Secondary  Task 


•  Tone  indicates  new 
message 

•  Click  ‘Acknowledge’  Button 
as  quickly  as  possible 
without  compromising  your 
primary  mission  objectives 

•  This  will  launch  the  chat 
window 

•  Relay  chat  text  (latitude  and 
longitude  Coordinates)  as 
quickly  and  as  accurately  as 
possible 

•  Click  ‘Send’  when  done 
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Task  Scenario  -  Notes 


•  Refuel  at  the  Base  only  in  an 
emergency  (requires  more 
time) 


•  Resolving  Window  Errors  - 
Move  cursor  to  bottom-right 
of  window  and  drag  right  and 
down 


Drag  from  here 


Assets 


KC-10  Refuel  Tanker 


68 


Task  Scenario  -  Priorities 


•  Destroy  as  many  hostile  aircraft  as 
quickly  as  possible. 

•  Do  not  allow  hostile  aircraft  to  enter 
friendly  territory  (Yellow  and  Red 
Zones ). 

•  Protect  the  Air  Base 

•  Protect  high  value  air  assets: 
AWACS,  Tanker 

•  Keep  as  many  fighters  airborne  for 
as  long  as  possible. 


Mig-25 


Workload 


•  See  Handout 
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Using  DDD  -  Questions 


•  Please  feel  free  to  ask  any  questions  about  how  to  use  the  task 

•  You  will  get  a  chance  to  run  through  a  practice  scenario  before  the  prior  to 
the  actual  experiment 


Sign-Up 


•  Please  take  a  few  minutes  to  complete  a  time  to  participate 


•  You  will  be  credited  in  Experimentrak  once  you  complete  the  testing  session 
•  PLEASE  NOTE!!!! 

If  you  cannot  make  it  to  your  scheduled  time,  call  407-823-1492  to 
reschedule  ASAP 
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